flow_id that mix signed and unsigned values no longer cause unnecessary table-slice splits. It also extends ocsf::derive to handle list-valued enum fields for full bidirectional OCSF enum normalization.
Apr 30, 2026 · @jachris, @mavam, @codex · #5354
ocsf::derive now derives OCSF enum sibling fields for lists, not just scalar enum fields. For example, DNS answers with flag_ids: [1, 3, 4] now also get flags: ["Authoritative Answer", "Recursion Desired", "Recursion Available"], and the reverse direction works for flags to flag_ids as well.
May 5, 2026 · @IyeOnline, @claude
Parsing data that mixes int64 and uint64 values in the same field no longer
produces unnecessary table-slice splits, improving batching performance. Fields
like flow_id that are always non-negative but occasionally exceed the signed
integer limit of 2^63 − 1 are now merged into a single uint64 column where
possible, instead of being emitted as separate slices.
May 6, 2026 · @mavam, @codex · #6128
Empty if branches no longer crash when running pipelines with the new executor. For example, if false {} now behaves like an empty pass-through branch instead of triggering an internal assertion failure.
Configured startup pipelines can now reference operators from static packages reliably. Previously, such pipelines could fail during node startup with module <package> not found, even though the same package operator worked when run manually after startup.
May 2, 2026 · @mavam, @codex · #6108
Package UDOs now load correctly when a typed string default looks like a TQL pattern, such as default: "/tmp-data/".
Previously, loading such a package could abort with an unexpected internal error before any pipeline ran.
]]>Apr 29, 2026 · @mavam, @codex · #6098
ClickHouse connection errors caused by TLS/plaintext mismatches now include the TLS notes and hint again. This helps identify when to_clickhouse is using TLS against a plaintext ClickHouse endpoint and suggests setting tls=false when appropriate.
Apr 28, 2026 · @tobim, @codex · #6086
Default retention policies now continue deleting metrics and diagnostics as their timestamps age into the retention window, even when older and newer events share a partition.
Previously, a partition that still contained newer events after retention could be skipped by later retention runs, leaving those events behind after they expired.
]]>Tenzir can now consume from and publish to NATS JetStream subjects with
from_nats and to_nats.
Use from_nats to receive one event per message. The raw payload appears in the
message blob field, and metadata_field attaches NATS metadata:
from_nats "alerts", metadata_field=nats
parsed = string(message).parse_json()
Use to_nats to publish one message per event. By default, the operator
serializes the whole event with this.print_ndjson():
from {severity: "high", alert_type: "suspicious-login"}
to_nats "alerts"
Both operators support configurable connection settings, authentication, and
the standard Tenzir tls record.
Apr 27, 2026 · @tobim, @codex · #6082
Static musl builds of tenzir no longer crash on deeply nested generated TQL
expressions.
This affected generated pipelines with deeply nested expressions, for example rules or transformations that expand into long left-associated operator chains.
The tenzir binary now links with a larger default thread stack size on musl,
which brings its behavior in line with non-static builds for these pipelines.
The from_http operator now supports paginate="odata" for
OData collection
responses such as Microsoft Graph:
from_http "https://graph.microsoft.com/v1.0/users",
headers={"ConsistencyLevel": "eventual"},
paginate="odata" {
read_json
}
This mode emits the objects from the response body's top-level value array
and follows top-level @odata.nextLink URLs until no next link is present. The
next link can be absolute or relative to the current response URL.
Apr 24, 2026 · @mavam, @codex · #6081
The to_sentinelone_data_lake operator now works in pipelines that run on the new executor. Previously, using it there failed before the pipeline could send events.
from {message: "hello"}
to_sentinelone_data_lake "https://example.com", token="TOKEN"
The subnet function now accepts typed IP addresses, plain IP strings, and
existing subnet values with an optional prefix length:
from {source_ip: 10.10.1.124}
net = subnet(source_ip, 24)
This returns 10.10.1.0/24 without converting the IP address to a string first.
When you omit the prefix, IPv4 addresses become /32 host subnets and IPv6
addresses become /128 host subnets.
The unroll operator no longer crashes when expanding very large lists into output that exceeds Arrow's per-array capacity.
The files operator now skips unreadable child directories during recursive traversal, emits a warning for each skipped directory by default, and continues listing accessible siblings. Set skip_permission_denied=true to ignore permission-denied paths silently: this suppresses warnings for skipped child directories and still makes an unreadable initial directory produce no events instead of an error. Non-permission filesystem errors continue to fail the pipeline.
Apr 23, 2026 · @IyeOnline
We fixed an issue in the context::enrich operator that did cause unbounded
memory growth.
Apr 23, 2026 · @tobim, @codex · #6068
Tenzir no longer segfaults on some very deep left-associated boolean
expressions in where clauses due to source-location handling.
Both list and record indexing in TQL now work with signed and unsigned integer indices.
This also applies to record field-position indexing and to the get function for records and lists.
Apr 20, 2026 · @tobim, @codex · #6059
Partition rebuilds now finish after persisting rebuilt partitions. Previously, rebuild jobs could remain stuck indefinitely even though the replacement partitions were written successfully.
The summarize operator now starts frequency-based emission with the first
input event and emits overdue periodic results before later events are
aggregated. This makes periodic output deterministic in reset,
cumulative, and update modes for delayed or sparse streams.
For example:
from {ts: 0ms.from_epoch(), x: 1},
{ts: 90ms.from_epoch(), x: 1},
{ts: 360ms.from_epoch(), x: 1}
delay ts
summarize count=count(), options={frequency: 300ms, mode: "cumulative"}
The first periodic result now consistently reports a count of 2 before the
third event arrives.
Apr 16, 2026 · @tobim, @codex · #6039
Tenzir nodes now honor standard HTTP proxy environment variables when connecting to Tenzir Platform:
HTTPS_PROXY=http://proxy.example:3128 tenzir-node
Use NO_PROXY to bypass the proxy for selected hosts. This helps deployments where outbound connections to the Platform websocket gateway must go through an HTTP proxy.
Apr 15, 2026 · @lava
We added a new operator to accept data from incoming HTTP connections.
The server option of the from_http operator is now deprecated.
Going forward, it should only be used for client-mode HTTP operations,
and the new accept_http operator should be used for server-mode
operations.
Apr 16, 2026 · @mavam, @codex · #6022
The hash_* functions now hash blob values by their raw bytes. This makes checksums computed from binary data match external tools such as md5sum and sha256sum.
For example:
from_file "trace.pcap" {
read_all binary=true
}
md5 = data.hash_md5()
This is useful for verifying file contents and round-tripping binary formats without leaving TQL.
]]>context::lookup operator, and it adds pipeline names to diagnostics and metrics for easier operational correlation. This release also improves export reliability under load and fixes Azure transport errors, HTTP Host headers for non-standard ports, and rebuilt-partition export correctness.
Apr 1, 2026 · @IyeOnline · #5964
The context::lookup operator enables unified matching of events against contexts
by combining live and retrospective filtering in a single operation.
The operator automatically translates context updates into historical queries while simultaneously filtering all newly ingested data against any context updates.
This provides:
live=trueretro=trueExample usage:
context::lookup "feodo", field=src_ip
where @name == "suricata.flow"
Mar 30, 2026 · @IyeOnline, @claude · #5959
The metrics and diagnostics operators now include a pipeline_name field.
Previously, output from these operators only identified the source pipeline by its ID. Now the human-readable name is available too, making it straightforward to filter or group results by pipeline name without needing to look up IDs separately.
Please keep in mind that pipeline names are not unique.
Apr 8, 2026 · @claude
Bumped Apache Arrow from 23.0.0 to 23.0.1, which includes an upstream fix
for unhandled Azure::Core::Http::TransportException in Arrow's
AzureFileSystem methods. Previously, transport-level errors (e.g., SSL
certificate failures) could crash the node during file listing, reading, or
writing. Additionally, the direct Azure SDK calls in the blob deletion code
paths now catch Azure::Core::RequestFailedException (the common base of
both StorageException and TransportException) instead of listing
specific exception types.
Apr 7, 2026 · @tobim, @codex · #5988
The export operator no longer emits partially populated events from rebuilt partitions when a row is null at the record level. Previously, some events could appear with most fields set to null while a few values, such as event_type or interface fields, were still present.
This makes exports from rebuilt data more reliable when investigating sparse or malformed-looking events.
Mar 31, 2026
The from_http and http operators now include the port in the Host header
when the URL uses a non-standard port. Previously, the port was omitted, which
caused requests to fail with HTTP 403 when the server validates the Host
header against the full authority, such as for pre-signed URL signature
verification.
The export command no longer fails or misses recent events when a node is flushing active partitions to disk under heavy load. Recent exports now keep the in-memory partitions they depend on alive until the snapshot completes, which preserves correctness for concurrent import and export workloads.
Feb 4, 2026 · @tobim, @codex · #5703
AWS operators now support OIDC-based authentication via the AssumeRoleWithWebIdentity API.
You can authenticate with AWS resources using OpenID Connect tokens from external identity providers like Azure, Google Cloud, or custom endpoints. This enables secure cross-cloud authentication without sharing long-lived AWS credentials.
Configure web identity authentication in any AWS operator by specifying a token source and target role:
from_s3 "s3://bucket/path", aws_iam={
region: "us-east-1",
assume_role: "arn:aws:iam::123456789012:role/cross-cloud-role",
web_identity: {
token_file: "/path/to/oidc/token"
}
}
The web_identity option accepts three token sources: token_file (path to a token file), token_endpoint (HTTP endpoint that returns a token), or token (direct token value). For HTTP endpoints, you can extract tokens from JSON responses using path.
Credentials automatically refresh before expiration, with exponential backoff retry logic for transient failures. This is especially useful for long-running pipelines that need persistent authentication.
Mar 30, 2026 · @jachris · #5954
Pipelines that use and, or, or if-else expressions run significantly faster in certain cases — up to 30× in our benchmarks. The improvement is most noticeable in pipelines with complex filtering or branching logic. No pipeline changes are needed to benefit.
Mar 23, 2026 · @mavam, @codex · #5939
The ocsf::derive operator now supports OCSF 1.8.0 events.
For example, you can now derive enum and sibling fields for events that declare
metadata.version: "1.8.0":
from {metadata: {version: "1.8.0"}, class_uid: 1007}
ocsf::derive
This keeps OCSF normalization pipelines working when producers emit 1.8.0
events.
Platform configuration validation now provides clearer error messages when an invalid configuration is encountered, helping you quickly diagnose and fix configuration issues.
Setting TENZIR_ENDPOINT to an unresolvable hostname no longer crashes the pipeline with a segfault.
Mar 25, 2026 · @mavam, @claude · #5949
ocsf::derive no longer emits a false warning when an _id field is set
to 99 (Other) and the sibling string contains a source-specific value.
Per the OCSF specification, 99/Other is an explicit escape hatch: the
integer signals that the value is not in the schema's enumeration and the
companion string must hold the raw value from the data source. For
example, the following is now accepted silently:
from {
metadata: { version: "1.7.0" },
type_uid: 300201,
class_uid: 3002,
auth_protocol_id: 99,
auth_protocol: "Negotiate",
}
ocsf::derive
Previously this produced a spurious warning: found invalid value for 'auth_protocol' because "Negotiate" is not a named enum caption.
Mar 24, 2026 · @lava
The Azure Blob Storage connector now handles Azure::Core::Http::TransportException
(e.g., SSL certificate errors) gracefully instead of crashing. Previously, a
self-signed certificate in the certificate chain would cause an unhandled
exception and terminate the node.
Mar 17, 2026 · @tobim
Feather store files now include a TENZIR:store:origin key in the Arrow table
schema metadata. The value is "ingest" for freshly ingested data, "rebuild"
for partitions created by the rebuild command, and "compaction" for partitions
created by the compaction plugin. This allows external tooling such as pyarrow
to distinguish how a partition was produced.
Mar 11, 2026 · @IyeOnline, @codex, @mavam, @raxyte · #5897
The to_clickhouse operator now supports dynamic table names via an expression
table=..., which must evaluate to a string. If the value is not a valid
table name, the events will be dropped with a warning.
With this change, the operator will also create a database if it does not exist.
The prime use-case for this are OCSF event streams:
subscribe "ocsf"
ocsf::cast encode_variants=true, null_fill=true
to_clickhouse table=f"ocsf.{class_name.replace(" ","_")}", ...
You can now install Tenzir on Apple Silicon macOS via Homebrew:
brew tap tenzir/tenzir
brew install --cask tenzir
You can also install directly without tapping first:
brew install --cask tenzir/tenzir/tenzir
The release workflow keeps the Homebrew cask in sync with the signed macOS package so installs and uninstalls stay current across releases.
The AWS Marketplace ECR repository tenzir-node was incorrectly populated with
the tenzir image. It now correctly ships tenzir-node, which runs a Tenzir
node by default.
If you relied on the previous behavior, you can restore it by setting tenzir
as a custom entrypoint in your ECS task definition.
The bundled Suricata schema now covers the remaining event types listed in the Suricata 8.0.3 EVE JSON format documentation: IKE (IKEv1/IKEv2), HTTP/2, PostgreSQL, and Modbus. This completes Suricata 8 schema coverage for Tenzir.
The read_syslog operator and parse_syslog function now accept RFC 5424 structured-data parameter names longer than 32 characters, which some vendors emit despite the specification limit.
For example, this message now parses successfully instead of being rejected:
<134>1 2026-03-18T11:00:51.194137+01:00 HOSTNAME abc 9043 23003147 [F5@12276 thx_f5_for_ignoring_the_32_char_limit_in_structured_data="thx"] broken example
This improves interoperability with vendor syslog implementations that exceed the RFC limit for structured-data parameter names.
Mar 14, 2026 · @aljazerzen · #5906
The batch timeout was only checked when a new event arrived, so a single event followed by an idle stream would never be emitted. The timeout now fires independently of upstream activity.
Mar 13, 2026 · @jachris · #5901
The parse_winlog function could fragment output into thousands of tiny
batches due to type conflicts in RenderingInfo/Keywords, where events with
one <Keyword> emitted a string but events with multiple emitted a list.
Additionally, EventData with unnamed <Data> elements is now always emitted
as a record with _0, _1, etc. as field names instead of a list.
Mar 12, 2026 · @jachris · #5899
The in operator for list expressions is up to 33x faster. Previously it
created and finalized entire Arrow arrays for every element comparison, causing
severe overhead for expressions like EventID in [5447, 4661, ...].
Additionally, comparing a typed null value with == now returns false instead
of null, and != returns true, fixing a correctness issue with null
handling in equality comparisons.
Mar 12, 2026 · @jachris · #5899
The in operator fast path now correctly prevents comparison of secret values.
Previously, secret_value in [...] would silently compare instead of returning
null with a warning, bypassing the established secret comparison policy.
Mar 12, 2026 · @jachris · #5900
Pattern equality checks now correctly consider the case-insensitive flag. Previously, two patterns that differed only in case sensitivity were treated as equal, violating the hash/equality contract.
Mar 12, 2026 · @jachris · #5899
Splitting Arrow arrays for string and blob types no longer over-reserves memory. Previously both output builders reserved the full input size each, using up to twice the necessary memory.
]]>Fixed a scheduling issue introduced in v5.24.0 that could cause the node to
become unresponsive when too many pipelines using detached operators like
from_udp were deployed simultaneously.
The bundled Suricata schema now includes types for four previously missing event types: ike, http2, pgsql, and modbus.
The ike type supports both IKEv1 and IKEv2 traffic. Version-specific fields are contained within dedicated ikev1 and ikev2 sub-objects, covering key exchange payloads, nonce payloads, client proposals, vendor IDs, and IKEv2 role/notify information.
The http2 type models HTTP/2 request and response streams including settings frames, header lists, error codes, and stream priority.
The pgsql type covers PostgreSQL session events with full request fields (simple queries, startup parameters, SASL authentication) and response fields (row counts, command completion, parameter status).
The modbus type captures industrial Modbus protocol transactions including function codes, access types, exception responses, diagnostic subfunctions, and MEI encapsulated interface data.
Mar 13, 2026 · @mavam, @codex · #5902
read_syslog and parse_syslog now extract a leading RFC 5424-style
structured-data block from RFC 3164 message content.
This pattern occurs in practice with some VMware ESXi messages, where
components such as Hostd emit a legacy syslog record and prepend structured
metadata before the human-readable message text.
For example, this raw syslog line:
<166>2026-02-11T18:01:45.587Z esxi-01.example.invalid Hostd[2099494]: [Originator@6876 sub=Vimsvc.TaskManager opID=11111111-2222-3333-4444-555555555555] Task Completed
now parses as:
{
facility: 20,
severity: 6,
timestamp: "2026-02-11T18:01:45.587Z",
hostname: "esxi-01.example.invalid",
app_name: "Hostd",
process_id: "2099494",
structured_data: {
"Originator@6876": {
sub: "Vimsvc.TaskManager",
opID: "11111111-2222-3333-4444-555555555555",
},
},
content: "Task Completed",
}
Events without extracted structured data keep the existing syslog.rfc3164
schema. Events with extracted structured data use
syslog.rfc3164.structured.
Mar 10, 2026 · @IyeOnline, @satta · #5888
The bundled Suricata schema now aligns with Suricata 8, enabling proper parsing and representation of events from Suricata 8 deployments.
This update introduces support for new event types including POP3, ARP, and BitTorrent DHT, along with enhancements to existing event types. QUIC events now include ja4 and ja4s fields for fingerprinting, DHCP events include vendor_class_identifier, and TLS certificate timestamps now use the precise time type instead of string representation.
These schema changes ensure that Tenzir can reliably ingest and process telemetry from Suricata 8 without data loss or type mismatches.
Mar 11, 2026 · @jachris · #5893
In some situations, pipelines could not be successfully started, leading to timeouts and a non-responsive node, especially during node start.
Mar 10, 2026 · @IyeOnline, @codex · #5886
Pipelines using chained list transforms such as xs.where(...).map(...).where(...) no longer trigger an internal assertion on sliced input batches.
Mar 9, 2026 · @mavam, @codex · #5877
Invalid Google Cloud credentials in from_google_cloud_pubsub no longer crash the node. Authentication errors now surface as operator diagnostics instead.
Mar 2, 2026 · @mavam, @codex · #5851
parse_syslog() and read_syslog now accept common Check Point structured-data variants that are not strictly RFC 5424 compliant. This includes key:"value" parameters, semicolon-separated parameters, and records that omit an SD-ID entirely.
For records without an SD-ID, Tenzir now normalizes the structured data under checkpoint_2620, so downstream pipelines can use a stable field path.
For example, the message <134>1 ... - [action:"Accept"; conn_direction:"Incoming"] now parses successfully and maps to structured_data.checkpoint_2620. This improves interoperability with Check Point exports and reduces ingestion-time preprocessing.
Mar 6, 2026 · @IyeOnline · #5805
JSON parsing errors now display the surrounding bytes at the error location. This makes it easier to diagnose malformed JSON in your data pipelines.
For example, if your JSON is missing a closing bracket, the error message shows you the bytes around that location and marks where the parser stopped expecting more input.
Mar 4, 2026 · @tobim, @codex · #5865
The load_tcp operator now makes DNS hostname resolution opt-in with the resolve_hostnames parameter (defaults to false).
Previously, the operator always attempted reverse DNS lookups for peer endpoints, which could fail in environments without working reverse DNS configurations. Now you can enable this behavior by setting resolve_hostnames to true:
load_tcp endpoint="0.0.0.0:5555" resolve_hostnames=true {
read_json
}
When enabled and DNS resolution fails, the operator emits a warning diagnostic (once) instead of failing. This allows the operator to continue functioning in environments where reverse DNS is unavailable or unreliable.
Mar 6, 2026 · @IyeOnline · #5805
We improved the reporting for unexpected diagnostics outside of operator execution, such as during startup. In these cases you will now get the diagnostic message.
Mar 6, 2026 · @IyeOnline · #5805
Fixed multiple issues that could cause errors or incorrect behavior when the schema of parsed events changes between records. This is particularly important when ingesting data from sources that may add, remove, or modify fields over time.
Schema mismatch warnings for repeated fields in JSON objects (which Tenzir interprets as lists) now include an explanatory hint, making it clearer what's happening when a field appears multiple times where a single value was expected.
]]>Mar 2, 2026 · @IyeOnline · #5855
We fixed a bug that could cause a crash when reading JSON data.
Mar 2, 2026 · @jachris · #5841
The CEF parser now handles unescaped = characters (which are not conforming to
the specification) by using a heuristic.
Feb 27, 2026 · @mavam, @codex · #5846
The new experimental hmac function computes Hash-based Message
Authentication Codes (HMAC) for strings and blobs. It supports SHA-256
(default), SHA-512, SHA-384, SHA-1, and MD5 algorithms.
Note: The key parameter is currently a plain string because function
arguments cannot be secrets yet. We plan to change this in the future.
from {
signature: hmac("hello world", "my-secret-key"),
}
{
signature: "90eb182d8396f16d4341d582047f45c0a97d73388c5377d9ced478a2212295ad",
}
Specify a different algorithm with the algorithm parameter:
from {
signature: hmac("hello world", "my-secret-key", algorithm="sha512"),
}
Feb 27, 2026 · @IyeOnline · #5842
We fixed a bug that would cause an assertion failure "Index error: array slice would exceed array length". This was introduced as part of an optimization in Tenzir Node v5.27.0.
]]>Feb 24, 2026 · @lava
Fixed in issue where the platform plugin did not correctly use the
configured certfile, keyfile and cafile options for client
certificate authentication, and improved the error messages for TLS
issues during platform connection.
Feb 22, 2026 · @mavam, @codex · #5819
The slice function now supports list types in addition to string. You can slice lists using the same begin, end, and stride parameters. Negative stride values are now supported for lists, letting you reverse or step backward through list data. String slicing continues to require a positive stride.
Example usage with lists:
[1, 2, 3, 4, 5].slice(begin=1, end=4) returns [2, 3, 4][1, 2, 3, 4, 5].slice(stride=-1) returns the list in reverse order[1, 2, 3, 4, 5].slice(begin=1, end=5, stride=-2) returns [5, 3]Feb 17, 2026 · @mavam, @codex · #5767
The sort function now supports two new parameters: desc for controlling
sort direction and cmp for custom comparison logic via binary lambdas.
Sort in descending order:
from {xs: [3, 1, 2]}
select ys = sort(xs, desc=true)
{ys: [3, 2, 1]}
Sort records by a specific field using a custom comparator:
from {xs: [{v: 2, id: "b"}, {v: 1, id: "a"}, {v: 2, id: "c"}]}
select ys = sort(xs, cmp=(left, right) => left.v < right.v)
{
ys: [
{v: 1, id: "a"},
{v: 2, id: "b"},
{v: 2, id: "c"},
],
}
The cmp lambda receives two elements and returns a boolean indicating whether
the first element should come before the second. Both parameters can be combined
to reverse a custom comparison.
Feb 17, 2026 · @tobim
The read_lines operator was accidently broken while it was ported
to the new execution API. This change restores its functionality.
HTTP header values containing colons are now parsed correctly.
]]>