This release adds Amazon Kinesis operators, regex Kafka topic selection, Google SecOps Chronicle import support, and inline drop_null_fields cleanup for cloud and streaming pipelines. It also improves HTTP server bind failures, package field defaults, and timeout flushing for OpenSearch and CloudWatch sinks.
🚀 Features
Section titled “🚀 Features”Selector defaults for UDO field parameters
Section titled “Selector defaults for UDO field parameters”Jun 12, 2026 · @mavam, @claude · #6352
Field-typed parameters of user-defined operators now accept selectors as defaults in the TQL frontmatter, such as this, this.name, or foo.bar. Previously, the only allowed default for a field parameter was null.
For example, this operator wraps a field into a record and operates on the entire event when no argument is given:
---args: named: - name: field type: field default: this---this = {wrapped: $field}Calling the operator without arguments wraps the whole event, while passing field=name wraps just that field. This makes it easy to write mapping operators that work on the full event by default but can be scoped to a subrecord on demand.
Regular expression topic selection for from_kafka
Section titled “Regular expression topic selection for from_kafka”Jun 8, 2026 · @mavam, @codex · #6262
The from_kafka operator can now consume from topics selected by a regular
expression when the topic argument starts with ^:
from_kafka "^tenant-.*\\.alerts$", offset="beginning"This lets one pipeline consume matching Kafka topics without listing each topic separately.
As part of this change, from_kafka now consumes all topics—exact and
regex—through Kafka’s consumer group subscription. This has two visible
effects. First, pipelines that share a group.id (default: tenzir) now split
the partitions of a topic among themselves instead of each receiving every
message; give each pipeline its own group.id to keep full copies. Second,
partition assignments now follow group rebalances, so they may shift when
pipelines with the same group.id start or stop. The exit option is not
available for regex topics, because a regex subscription has no bounded set of
partitions to reach the end of.
drop_null_fields function
Section titled “drop_null_fields function”Jun 5, 2026 · @zedoraps, @claude · #6261
The new drop_null_fields function strips fields whose value is null from a
record, mirroring the existing drop_null_fields operator. This lets you clean
optional fields inline in expressions, for example
from_http "...", body=drop_null_fields({license: $license, version: 1, mapping: $mapping}).
Amazon Kinesis operators
Section titled “Amazon Kinesis operators”May 15, 2026 · @mavam, @codex · #6188
Tenzir can now read from and write to Amazon Kinesis data streams with the from_amazon_kinesis and to_amazon_kinesis operators:
from_amazon_kinesis "telemetry", start="trim_horizon"this = string(message).parse_json()read_ndjson// ... transform events ...to_amazon_kinesis "telemetry", partition_key=src_ipThe operators support existing AWS IAM configuration and endpoint overrides for local testing environments such as LocalStack.
🔧 Changes
Section titled “🔧 Changes”Google SecOps Chronicle import API
Section titled “Google SecOps Chronicle import API”Jun 3, 2026 · @raxyte, @codex · #6216
The to_google_secops operator now uses the Chronicle logs.import,
events.import, and entities.import APIs instead of the legacy unstructured
ingestion API.
The operator now targets a SecOps instance with project, region, and
instance, authenticates with Google Cloud OAuth2 credentials, and supports raw
log ingestion with mode="raw_log", UDM event ingestion with mode="udm_event", and
entity ingestion with mode="udm_entity".
🐞 Bug Fixes
Section titled “🐞 Bug Fixes”Timeout flushing for OpenSearch and CloudWatch sinks
Section titled “Timeout flushing for OpenSearch and CloudWatch sinks”Jun 12, 2026 · @mavam, @codex · #6188
The to_opensearch and to_amazon_cloudwatch sinks now reliably flush partial batches after their configured timeout while the pipeline continues to run.
Previously, concurrent wakeups could race with batch deadline updates, which could cause timeout-based flushes or CloudWatch send-completion handling to be missed.
Parallel signed AWS HTTP requests
Section titled “Parallel signed AWS HTTP requests”Jun 12, 2026 · @mavam, @codex · #6188
AWS-backed operators that issue parallel signed HTTP requests now send concurrent requests over separate pooled connections.
Previously, HTTP/1.1 request pipelining could serialize parallel requests behind a single connection, so parallel settings for sinks such as to_amazon_cloudwatch did not provide the intended concurrency.
HTTP server operators no longer crash the node on bind failure
Section titled “HTTP server operators no longer crash the node on bind failure”Jun 9, 2026 · @Zedoraps · #6267
The accept_http, accept_opensearch, and serve_http operators no longer
abort the node when they fail to bind their endpoint. Starting a second
pipeline on a port that is already in use—or restarting one before the
previous instance has released its socket—previously crashed the entire node
process. Now the operator emits a regular diagnostic:
error: failed to start HTTP server: failed to bind to async server socket:0.0.0.0:8774: Address already in useThe pipeline that hit the conflict exits with an error while every other pipeline running on the node keeps going.
Nested assignment through package UDO field parameters
Section titled “Nested assignment through package UDO field parameters”Jun 8, 2026 · @mavam, @jachris, @codex · #6264
Package UDO field parameters now support assignment through field accesses. This lets package operators write to nested fields of a field argument using dot syntax, string-literal bracket syntax, or dynamic index expressions.
For example, this package UDO body is now valid:
$field.subfield = $value$field["quoted-field"] = $value$field[$key] = $valueThis is useful for mapping operators that accept an event= target and need to keep temporary state under that target instead of creating top-level scratch fields.