Mar 13, 2026 · @mavam, @codex · #5902
read_syslog and parse_syslog now extract a leading RFC 5424-style
structured-data block from RFC 3164 message content.
This pattern occurs in practice with some VMware ESXi messages, where
components such as Hostd emit a legacy syslog record and prepend structured
metadata before the human-readable message text.
For example, this raw syslog line:
<166>2026-02-11T18:01:45.587Z esxi-01.example.invalid Hostd[2099494]: [Originator@6876 sub=Vimsvc.TaskManager opID=11111111-2222-3333-4444-555555555555] Task Completed
now parses as:
{
facility: 20,
severity: 6,
timestamp: "2026-02-11T18:01:45.587Z",
hostname: "esxi-01.example.invalid",
app_name: "Hostd",
process_id: "2099494",
structured_data: {
"Originator@6876": {
sub: "Vimsvc.TaskManager",
opID: "11111111-2222-3333-4444-555555555555",
},
},
content: "Task Completed",
}
Events without extracted structured data keep the existing syslog.rfc3164
schema. Events with extracted structured data use
syslog.rfc3164.structured.
Mar 10, 2026 · @IyeOnline, @satta · #5888
The bundled Suricata schema now aligns with Suricata 8, enabling proper parsing and representation of events from Suricata 8 deployments.
This update introduces support for new event types including POP3, ARP, and BitTorrent DHT, along with enhancements to existing event types. QUIC events now include ja4 and ja4s fields for fingerprinting, DHCP events include vendor_class_identifier, and TLS certificate timestamps now use the precise time type instead of string representation.
These schema changes ensure that Tenzir can reliably ingest and process telemetry from Suricata 8 without data loss or type mismatches.
Mar 11, 2026 · @jachris · #5893
In some situations, pipelines could not be successfully started, leading to timeouts and a non-responsive node, especially during node start.
Mar 10, 2026 · @IyeOnline, @codex · #5886
Pipelines using chained list transforms such as xs.where(...).map(...).where(...) no longer trigger an internal assertion on sliced input batches.
Mar 9, 2026 · @mavam, @codex · #5877
Invalid Google Cloud credentials in from_google_cloud_pubsub no longer crash the node. Authentication errors now surface as operator diagnostics instead.
Mar 2, 2026 · @mavam, @codex · #5851
parse_syslog() and read_syslog now accept common Check Point structured-data variants that are not strictly RFC 5424 compliant. This includes key:"value" parameters, semicolon-separated parameters, and records that omit an SD-ID entirely.
For records without an SD-ID, Tenzir now normalizes the structured data under checkpoint_2620, so downstream pipelines can use a stable field path.
For example, the message <134>1 ... - [action:"Accept"; conn_direction:"Incoming"] now parses successfully and maps to structured_data.checkpoint_2620. This improves interoperability with Check Point exports and reduces ingestion-time preprocessing.
Mar 6, 2026 · @IyeOnline · #5805
JSON parsing errors now display the surrounding bytes at the error location. This makes it easier to diagnose malformed JSON in your data pipelines.
For example, if your JSON is missing a closing bracket, the error message shows you the bytes around that location and marks where the parser stopped expecting more input.
Mar 4, 2026 · @tobim, @codex · #5865
The load_tcp operator now makes DNS hostname resolution opt-in with the resolve_hostnames parameter (defaults to false).
Previously, the operator always attempted reverse DNS lookups for peer endpoints, which could fail in environments without working reverse DNS configurations. Now you can enable this behavior by setting resolve_hostnames to true:
load_tcp endpoint="0.0.0.0:5555" resolve_hostnames=true {
read_json
}
When enabled and DNS resolution fails, the operator emits a warning diagnostic (once) instead of failing. This allows the operator to continue functioning in environments where reverse DNS is unavailable or unreliable.
Mar 6, 2026 · @IyeOnline · #5805
We improved the reporting for unexpected diagnostics outside of operator execution, such as during startup. In these cases you will now get the diagnostic message.
Mar 6, 2026 · @IyeOnline · #5805
Fixed multiple issues that could cause errors or incorrect behavior when the schema of parsed events changes between records. This is particularly important when ingesting data from sources that may add, remove, or modify fields over time.
Schema mismatch warnings for repeated fields in JSON objects (which Tenzir interprets as lists) now include an explanatory hint, making it clearer what's happening when a field appears multiple times where a single value was expected.
]]>Mar 2, 2026 · @IyeOnline · #5855
We fixed a bug that could cause a crash when reading JSON data.
Mar 2, 2026 · @jachris · #5841
The CEF parser now handles unescaped = characters (which are not conforming to
the specification) by using a heuristic.
Feb 27, 2026 · @mavam, @codex · #5846
The new experimental hmac function computes Hash-based Message
Authentication Codes (HMAC) for strings and blobs. It supports SHA-256
(default), SHA-512, SHA-384, SHA-1, and MD5 algorithms.
Note: The key parameter is currently a plain string because function
arguments cannot be secrets yet. We plan to change this in the future.
from {
signature: hmac("hello world", "my-secret-key"),
}
{
signature: "90eb182d8396f16d4341d582047f45c0a97d73388c5377d9ced478a2212295ad",
}
Specify a different algorithm with the algorithm parameter:
from {
signature: hmac("hello world", "my-secret-key", algorithm="sha512"),
}
Feb 27, 2026 · @IyeOnline · #5842
We fixed a bug that would cause an assertion failure "Index error: array slice would exceed array length". This was introduced as part of an optimization in Tenzir Node v5.27.0.
]]>Feb 24, 2026 · @lava
Fixed in issue where the platform plugin did not correctly use the
configured certfile, keyfile and cafile options for client
certificate authentication, and improved the error messages for TLS
issues during platform connection.
Feb 22, 2026 · @mavam, @codex · #5819
The slice function now supports list types in addition to string. You can slice lists using the same begin, end, and stride parameters. Negative stride values are now supported for lists, letting you reverse or step backward through list data. String slicing continues to require a positive stride.
Example usage with lists:
[1, 2, 3, 4, 5].slice(begin=1, end=4) returns [2, 3, 4][1, 2, 3, 4, 5].slice(stride=-1) returns the list in reverse order[1, 2, 3, 4, 5].slice(begin=1, end=5, stride=-2) returns [5, 3]Feb 17, 2026 · @mavam, @codex · #5767
The sort function now supports two new parameters: desc for controlling
sort direction and cmp for custom comparison logic via binary lambdas.
Sort in descending order:
from {xs: [3, 1, 2]}
select ys = sort(xs, desc=true)
{ys: [3, 2, 1]}
Sort records by a specific field using a custom comparator:
from {xs: [{v: 2, id: "b"}, {v: 1, id: "a"}, {v: 2, id: "c"}]}
select ys = sort(xs, cmp=(left, right) => left.v < right.v)
{
ys: [
{v: 1, id: "a"},
{v: 2, id: "b"},
{v: 2, id: "c"},
],
}
The cmp lambda receives two elements and returns a boolean indicating whether
the first element should come before the second. Both parameters can be combined
to reverse a custom comparison.
Feb 17, 2026 · @tobim
The read_lines operator was accidently broken while it was ported
to the new execution API. This change restores its functionality.
HTTP header values containing colons are now parsed correctly.
]]>Feb 12, 2026 · @mavam, @claude · #5753
User-defined operators in packages can now declare optional field-type parameters with null as the default value. This allows operators to accept field selectors that are not required to be provided.
When a field parameter is declared with type: field and default: null, you can omit the argument when calling the operator, and the parameter will receive a null value instead. You can then check whether a field was provided by comparing the parameter to null within the operator definition.
Example:
In your package's operator definition, declare an optional field parameter:
args:
named:
- name: selector
type: field
default: null
In the operator implementation, check if the field was provided:
set result = if $selector != null then "field provided" else "field omitted"
When calling the operator, the field argument becomes optional:
my_operator # field is null
my_operator selector=x.y # field is x.y
Only null is allowed as the default value for field parameters. Non-null defaults are rejected with an error during package loading.
Feb 11, 2026 · @mavam, @claude
The paginate parameter for the from_http and http operators now supports link-based pagination via the Link HTTP header.
Previously, pagination was only available through a lambda function that extracted the next URL from response data. Now you can use paginate="link" to automatically follow pagination links specified in the response's Link header, following RFC 8288. This is useful for APIs that use HTTP header-based pagination instead of embedding next URLs in the response body.
The operator parses the Link header and follows the rel=next relation to automatically fetch the next page of results.
Example:
from_http "https://api.example.com/data", paginate="link"
If an invalid pagination mode is provided (neither a lambda nor "link"), the operator now reports a clear error message.
Feb 6, 2026 · @mavam, @claude · #5721, #5738
The from_mysql operator lets you read data directly from MySQL databases.
Read a table:
from_mysql table="users", host="localhost", port=3306, user="admin", password="secret", database="mydb"
List tables:
from_mysql show="tables", host="localhost", port=3306, user="admin", password="secret", database="mydb"
Show columns:
from_mysql table="users", show="columns", host="localhost", port=3306, user="admin", password="secret", database="mydb"
And ultimately execute a custom SQL query:
from_mysql sql="SELECT id, name FROM users WHERE active = 1",
host="localhost",
port=3306,
user="admin",
password="secret",
database="mydb"
The operator supports TLS/SSL connections for secure communication with MySQL
servers. Use tls=true for default TLS settings, or pass a record for
fine-grained control:
from_mysql table="users", host="db.example.com", database="prod", tls={
cacert: "/path/to/ca.pem",
certfile: "/path/to/client-cert.pem",
keyfile: "/path/to/client-key.pem",
}
The operator supports MySQL's caching_sha2_password authentication method and automatically maps MySQL data types to Tenzir types.
Use live=true to continuously stream new rows from a table. The operator
tracks progress using a watermark on an integer column, polling for rows above
the last-seen value:
from_mysql table="events", live=true, host="localhost", database="mydb"
By default, the tracking column is auto-detected from the table's auto-increment primary key. To specify one explicitly:
from_mysql table="events", live=true, tracking_column="event_id",
host="localhost", database="mydb"
Feb 12, 2026 · @IyeOnline
We have significantly improved the performance of the write_lines operator.
Feb 12, 2026 · @mavam, @claude · #5752
User-defined operators in packages can now declare parameters with the secret type to ensure that secret values are properly handled as secret expressions:
args:
positional:
- name: api_key
type: secret
description: "API key to use for authentication"
Feb 5, 2026 · @mavam, @claude · #5728
The merge() function now performs a recursive deep merge when merging two records. Previously, nested fields were dropped when merging, so merge({hw: {sn: "XYZ123"}}, {hw: {model: "foobar"}}) would incorrectly produce {hw: {model: "foobar"}} instead of recursively merging the nested fields. The function now correctly produces {hw: {sn: "XYZ123", model: "foobar"}} by materializing both input records and performing a deep merge on them.
Feb 3, 2026 · @mavam, @claude · #5715
The sigma operator now correctly loads all rules when given a directory containing multiple Sigma rule files. Previously, only the last processed rule file would be retained because the rules collection was being cleared on every recursive directory traversal.
sigma "/path/to/sigma/rules"
All rules found in the directory and its subdirectories will now be loaded and used to match against input events.
]]>where, replace, and if operators, along with Kafka decompression support and a new raw_message option for the read_syslog operator.
Jan 27, 2026 · @mavam, @claude · #5687
The read_syslog operator now supports a raw_message parameter that preserves the original, unparsed syslog message in a field of your choice. This is useful when you need to retain the exact input for auditing, debugging, or compliance purposes.
When you specify raw_message=<field>, the operator stores the complete input message (including all lines for multiline messages) in the specified field. This works with all syslog formats, including RFC 5424, RFC 3164, and octet-counted messages.
For example:
read_syslog raw_message=original_input
This stores the unparsed message in the original_input field alongside the parsed structured fields like hostname, app_name, message, and others.
Jan 30, 2026 · @jachris · #5701
The condition of if statements is no longer erroneously evaluated early when
it contains a lambda expression that references runtime fields.
Jan 30, 2026 · @raxyte, @claude · #5697
Kafka connectors now support decompressing messages with zstd, lz4 and gzip.
Jan 29, 2026 · @jachris, @claude · #5698
The JSON parser no longer intermittently fails with "The input is not valid UTF-8" when parsing data containing multi-byte UTF-8 characters such as accented letters or emojis.
Jan 29, 2026 · @mavam, @claude · #5696
The replace operator no longer triggers an assertion failure when using
with=null on data processed by operators like ocsf::cast.
load_file "dns.json"
read_json
ocsf::cast "dns_activity"
replace what="", with=null
Jan 28, 2026 · @jachris, @claude
The where operator optimization now correctly handles optional fields marked with ?. Previously, the optimizer didn't account for the optional marker, which could result in incorrect query optimization. This fix ensures that optional field accesses are handled properly without affecting the optimization of regular field accesses.
Jan 22, 2026 · @tobim, @claude · #5605
The summarize operator now supports periodic emission of aggregation results at fixed intervals, enabling real-time streaming analytics and monitoring use cases.
Use the options named argument with frequency to emit results every N seconds:
summarize count(this), src_ip, options={frequency: 5s}
This emits aggregation results every 5 seconds, showing the count per source IP for events received during each interval:
{src_ip: 192.168.1.1, count: 42}
{src_ip: 192.168.1.2, count: 17}
// ... 5 seconds later ...
{src_ip: 192.168.1.1, count: 38}
{src_ip: 192.168.1.3, count: 9}
The mode parameter controls how aggregations behave across emissions:
Reset mode (default) resets aggregations after each emission, providing per-interval metrics:
summarize sum(bytes), options={frequency: 10s}
// Shows bytes per 10-second window
Cumulative mode accumulates values across emissions, providing running totals:
summarize sum(bytes), options={frequency: 10s, mode: "cumulative"}
// Shows total bytes seen so far
Update mode emits only when values change from the previous emission, reducing output noise in monitoring scenarios:
summarize count(this), severity, options={frequency: 1s, mode: "update"}
// Emits only when the count for a severity level changes
The operator always emits final results when the input stream ends, ensuring no data is lost.
Jan 21, 2026 · @tobim, @claude · #5675
The load_sqs, save_sqs, from_s3, to_s3, from_kafka, and to_kafka operators now support AWS IAM authentication through a new aws_iam option. You can configure explicit credentials, assume IAM roles, use AWS CLI profiles, or rely on the default credential chain.
The aws_iam option accepts these fields:
profile: AWS CLI profile name for credential resolutionaccess_key_id: AWS access key IDsecret_access_key: AWS secret access keysession_token: AWS session token for temporary credentialsassume_role: IAM role ARN to assumesession_name: Session name for role assumptionexternal_id: External ID for role assumptionAdditionally, the SQS and Kafka operators accept a top-level aws_region option:
load_sqs and save_sqs: Configures the AWS SDK client region for queue URL resolutionfrom_kafka and to_kafka: Required for MSK authentication (used to construct the authentication endpoint URL)You can also combine explicit credentials with role assumption. This uses the provided credentials to call STS AssumeRole and obtain temporary credentials for the assumed role:
load_sqs "my-queue", aws_iam={
access_key_id: "AKIAIOSFODNN7EXAMPLE",
secret_access_key: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
assume_role: "arn:aws:iam::123456789012:role/my-role"
}
For example, to load from SQS with a specific region:
load_sqs "my-queue", aws_region="us-east-1", aws_iam={
access_key_id: "AKIAIOSFODNN7EXAMPLE",
secret_access_key: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}
To use an AWS CLI profile:
load_sqs "my-queue", aws_iam={
profile: "production"
}
To assume an IAM role:
from_s3 "s3://bucket/path", aws_iam={
assume_role: "arn:aws:iam::123456789012:role/my-role",
session_name: "tenzir-session",
external_id: "unique-id"
}
For Kafka MSK authentication, the aws_region option is required:
from_kafka "my-topic", aws_region="us-east-1", aws_iam={
profile: "production"
}
When no explicit credentials or profile are configured, operators use the AWS SDK's default credential provider chain, which checks environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY), AWS configuration files (~/.aws/credentials), EC2/ECS instance metadata, and other standard sources. This applies both when aws_iam is omitted entirely and when aws_iam is specified without access_key_id, secret_access_key, or profile.
Jan 21, 2026 · @mavam, @claude
The parse_syslog function now supports RFC 6587 octet-counted framing, where syslog messages are prefixed with their byte length (for example, 65 <syslog-message>). This framing is commonly used in TCP-based syslog transport to handle message boundaries.
The new octet_counting parameter for parse_syslog offers three modes:
octet_counting=true: Require a length prefix. Emits a warning and returns null if the input lacks a valid prefix.octet_counting=false: Never strip a length prefix. Parse the input as-is.Jan 12, 2026 · @IyeOnline · #5646
We have added support for per-actor/per-thread allocation tracking. When enabled, these stats will track which actor (or thread) allocated how much memory. This gives much more detailed insights into where memory is allocated. By default these detailed statistics are not collected, as they introduce a cost to every allocation.
Jan 20, 2026 · @mavam, @claude · #5673
The ocsf::derive operator now preserves original field order instead of
reordering alphabetically. Derived enum/sibling pairs are inserted at the
position of the first field, ordered alphabetically within each pair (e.g.,
activity_id before activity_name). Non-OCSF fields remain at their original
positions.
For example, given the input:
{foo: 1, class_uid: 1001}
The output is now:
{foo: 1, class_name: "...", class_uid: 1001}
Previously, the output was alphabetically sorted:
{class_name: "...", class_uid: 1001, foo: 1}
Jan 19, 2026 · @jachris · #5670
The from_s3 and from_azure_blob_storage operators now also delete existing
directory marker objects along the glob path when remove=true. Directory
markers are zero-byte objects with keys ending in / that some cloud storage
tools create. These artifacts can accumulate over time, increasing API costs and
slowing down listing operations.
Jan 23, 2026 · @raxyte · #5677
The from_http server now has a stable memory usage when used with a slow
downstream, especially in situations where the client timeouts and retries
requests.
Jan 22, 2026 · @jachris, @claude · #5680
In rare cases, a phantom pipeline with an empty ID could appear in the pipeline list that couldn't be deleted through the API.
Jan 12, 2026 · @raxyte, @codex · #5654
The from_kafka operator now commits offsets per partition and tracks partition
EOFs based on the current assignment, preventing premature exits and
cross-partition replays after restarts.
Deleting files from S3 or Azure Blob Storage via from_s3 or
from_azure_blob_storage with the remove=true option no longer creates empty
directory marker objects in the parent directory when the last file of the
directory is deleted.
parse_xml and parse_winlog) for analyzing XML-formatted logs including Windows Event Logs. It also introduces the parallel operator for parallel pipeline execution, fixes a socket leak in from_http that could cause resource exhaustion, and includes several stability fixes for gRPC operators and the pipeline API.
Jan 5, 2026 · @jachris · #5644
The new tenzir.metrics.operator_buffers metrics track the total bytes and
events buffered across all operators of a pipeline. The metrics are emitted
every second and include:
timestamp: The point in time when the data was recordedpipeline_id: The pipeline's unique identifierbytes: Total bytes currently bufferedevents: Total events currently buffered (for events only)Use metrics "operator_buffers" to access these metrics.
Dec 30, 2025 · @mavam, @claude · #5640, #5645
The new parse_xml and parse_winlog functions parse XML strings into structured records, enabling analysis of XML-formatted logs and data sources.
The parse_xml function offers flexible XML parsing with XPath-based element selection, configurable attribute handling, namespace management, and depth limiting. It supports multiple match results as lists and handles both simple and complex XML structures.
The parse_winlog function specializes in parsing Windows Event Log XML format, automatically finding Event elements and transforming EventData/UserData sections into properly structured fields.
Both functions integrate with Tenzir's multi-series builder for schema inference and type handling.
Dec 23, 2025 · @raxyte · #5632
The parallel operator executes a pipeline across multiple parallel pipeline
instances to improve throughput for computationally expensive operations. It
automatically distributes input events across the pipeline instances and merges
their outputs back into a single stream.
Use the jobs parameter to specify how many pipeline instances to spawn.
For example, to parse JSON in parallel across 4 pipeline instances:
from_file "input.ndjson"
read_lines
parallel 4 {
this = line.parse_json()
}
Jan 12, 2026 · @raxyte, @claude · #5652
Repeated warnings and errors now resurface every 4 hours instead of being suppressed forever. Previously, once a diagnostic was shown, it would never appear again even if the underlying issue persisted. This change helps users notice recurring problems that may require attention.
The throttle operator now rate-limits events instead of bytes. Use the rate
option to specify the maximum number of events per window, weight to assign
custom per-event weights, and drop to discard excess events instead of
waiting. The operator also emits metrics for dropped events.
Jan 15, 2026 · @jachris · #5651
Previously, it was possible for the node to enter a state where the internal pipeline API was no longer responding, thus rendering the platform unresponsive.
Jan 14, 2026 · @mavam, @claude · #5661
We fixed bugs in several gRPC-based operators:
from_velociraptor on shutdown.to_google_cloud_pubsub on
shutdown.from_google_cloud_pubsub that could cause a crash.Jan 14, 2026 · @raxyte · #5660
The export operator incorrectly skipped partitions when evaluating in
predicates with uncertain membership. This caused queries like
export | where field in [values...] to potentially miss matching events.
Jan 14, 2026 · @IyeOnline · #5659
We fixed a bug in from_kafka that would cause it to not produce events.
Jan 8, 2026 · @jachris, @claude · #5647
The from_http operator sometimes left sockets in CLOSE_WAIT state instead of
closing them properly. This could lead to resource exhaustion on long-running
nodes receiving many HTTP requests.
Jan 7, 2026 · @tobim, @claude · #5649
The format_time and parse_time functions in the static binary now correctly
use the operating system's timezone database.
Dec 23, 2025 · @raxyte · #5632
The every and cron operators now correctly propagate errors from their subpipelines instead of silently swallowing them.
Dec 31, 2025 · @raxyte, @codex
Expression evaluation could produce a length mismatch when processing heterogeneous data, potentially causing assertion failures. This affected various operations including binary and unary operators, field access, indexing, and aggregation functions.
Dec 30, 2025 · @raxyte
The deduplicate operator with count_field option could cause assertion failures when discarding events.
Dec 29, 2025 · @raxyte · #5637
The save_tcp connector now gracefully shuts down on pipeline stop and
connection failures. Previously, the connector could abort the entire
application on exit.
Dec 23, 2025 · @jachris · #5629
Compaction rules can now use event timestamps instead of import time when selecting data by age. Configure this using the new optional field key in the compaction configuration.
Previously, compaction always used the import time to determine which partitions to compact. Now you can specify any timestamp field from your events:
tenzir:
compaction:
time:
rules:
- name: compact-old-logs
after: 7d
field: timestamp # Use event timestamp instead of import time
pipeline: |
summarize count=count(), src_ip
When field is not specified, compaction continues to use import time for backward compatibility.
Dec 22, 2025 · @raxyte · #5622
The deduplicate operator now supports a count_field option that adds a field
to each output event showing how many events were dropped for that key.
Example
from {x: 1, seq: 1}, {x: 1, seq: 2}, {x: 1, seq: 3}, {x: 1, seq: 4}
deduplicate x, distance=2, count_field=drop_count
{x: 1, seq: 1, drop_count: 0}
{x: 1, seq: 4, drop_count: 2}
Events that are the first occurrence of a key or that trigger output after
expiration have a count of 0.
The Tenzir Node now lets you configure the minimum TLS version and TLS ciphers accepted for the connection to the Tenzir Platform:
plugins:
platform:
tls-min-version: "1.2"
tls-ciphers: "HIGH:!aNULL:!MD5"
Dec 17, 2025 · @IyeOnline · #5559
All operators and connectors that use TLS now support centralized node-level
configuration. Instead of passing TLS options to each operator individually,
you can configure them once in tenzir.yaml under tenzir.tls.
Arguments passed directly to the operator itself via an argument take precedence over the configuration entry.
The following options are available:
enable: Enable TLS on all operators that support itskip-peer-verification: Disable certificate verificationcacert: Path to a CA certificate bundle for server verificationcertfile: Path to a client certificate filekeyfile: Path to a client private key filetls-min-version: Minimum TLS protocol version ("1.0", "1.1", "1.2", or "1.3")tls-ciphers: OpenSSL cipher list stringThe later two options have also been added as operator arguments.
For server-mode operators (load_http server=true, load_tcp), mutual TLS (mTLS)
authentication is now supported:
tls-client-ca: Path to a CA certificate for validating client certificatestls-require-client-cert: Require clients to present valid certificatesThese two options are also available as operator arguments.
Example configuration enforcing TLS 1.2+ with specific ciphers:
tenzir:
tls:
tls-min-version: "1.2"
tls-ciphers: "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256"
cacert: "/etc/ssl/certs/ca-certificates.crt"
Dec 23, 2025 · @jachris · #5629
The default compaction rules for tenzir.metrics.* and tenzir.diagnostic events now correctly use the timestamp field instead of import time.
Previously, these built-in compaction rules relied on import time to determine which events to compact, which could lead to inconsistent results as the import time is not computed per-event. As a result, it was possible that metrics and diagnostics were not deleted even though they expired.
]]>Dec 19, 2025 · @IyeOnline · #5612
We fixed an assertion failure and subsequent crash that could occur when parsing events that contain duplicate keys.
Dec 16, 2025 · @IyeOnline · #5612
We resolved an issue that would appear when reading in lists (e.g. JSON [])
where the elements had different types. Tenzir's type system at this time only
supports storing a single type in a list. Our parsers resolve this issue by first
attempting conversions (e.g. to a common numeric type) and turning all values
into strings as a last resort. Previously this would however also break Tenzir's
batch processing leading to significant performance loss. This has now been fixed.
publish operator could drop events.
Dec 18, 2025 · @raxyte · #5618
The publish operator could drop events during flush operations.
Events are now reliably delivered to subscribers.
Dec 17, 2025 · @tobim
User-defined operators in packages can now declare arguments in their YAML frontmatter, enabling parameterized operator definitions with the same calling convention as built-in operators.
Arguments can be positional or named. Both support optional default values and can be called with literals, constant expressions, or dynamically evaluated runtime expressions such as fields.
For example, create a reusable operator to set fields dynamically:
---
description: "Set a field to a value"
args:
positional:
- name: field
type: field
- name: value
type: string
named:
- name: prefix
type: string
default: ""
---
$field = $prefix + $value
Use the operator with both constant and runtime arguments:
from {x: 1}
mypkg::set_field this.name, "Alice", prefix="User: "
{
x: 1,
name: "User: Alice",
}
Parameters can be typed with a type name passed in the namesake field. In case
the passed in expression can be evaluated at instantiation time it is checked
against the type and a diagnostic is returned if it does not match. In case a
type check is not possible because the expression contains references to
run-time data, the type check is omitted, and potential errors will be flagged at
runtime. Field-path arguments (declared via type: field) accept selectors and
cannot declare defaults.
Dec 16, 2025 · @raxyte · #5611
The from_file, from_s3, from_gcs, and from_azure_blob_storage operators
now support an optional max_age parameter that filters files based on their
last modification time. Only files modified within the specified duration from
now will be processed.
Example
Process only files modified in the last hour:
from_file "/var/log/security/*.json", max_age=1h
Dec 15, 2025 · @IyeOnline · #5593
We have improved our Google Cloud PubSub integration with the addition of the new
from_google_cloud_pubsub and to_google_cloud_pubsub operators.
These operators are direct void -> event and event -> void operators, which means that they ensure a 1:1 relation between events and messages.
The from_google_cloud_pubsub operator can also attach metadata such
as message ID, publish time, and attributes for downstream enrichment.
The legacy load_google_cloud_pubsub and save_google_cloud_pubsub operators
are deprecated in favor of these event-preserving counterparts.
Dec 12, 2025 · @raxyte · #5601
The from_http operator in server mode now implements backpressure, waiting for
each request to be processed before accepting new data. This prevents memory
pressure during traffic spikes from webhook integrations or log receivers.
A new max_connections parameter limits simultaneous connections:
from_http "0.0.0.0:8080", server=true, max_connections=50
The default is 10 connections. Additional connections are rejected until a slot frees up, keeping your pipelines stable under heavy load.
Dec 12, 2025 · @raxyte · #5599
The new from_sentinelone_data_lake operator allows you to query the SentinelOne
Singularity Data Lake using PowerQuery and retrieve security events directly into
your Tenzir pipelines. Tenzir's integrations with SentinelOne now allow you to
send data to and load data from SentinelOne Data Lakes.
Example
Query threat events and filter by severity:
from_sentinelone_data_lake "https://xdr.eu1.sentinelone.net",
token=secret("sentinelone-token"),
query="severity > 3 | columns id",
start=now()-7d
The operator sends a request to the /api/powerQuery endpoint with optional time
range filters and parses the tabular response into events for downstream processing.
Dec 8, 2025 · @IyeOnline · #5445
Our parsers now have improved support for repeated keys in a an event. Previously a later key-value pair would always overwrite the previous one. With this change the value is transparently upgraded to a list of values.
The new from_kafka operator allows you to receive one event per Kafka message,
thus keeping the event boundary unlike load_kafka, which has now been
deprecated.
Example
Use from_kafka to parse JSON events from a topic:
from_kafka "events"
this = message.parse_json()
The to_google_cloud_logging operator now checks for the GOOGLE_CLOUD_PROJECT environment variable
if no explicit project id is given, before falling back to the Google Metadata service.
Oct 27, 2025 · @tobim
The tenzir binary is now bundled directly with the tenzir Python wheel.
This means you can run Tenzir pipelines on any machine with uv installed,
without any separate installation steps.
Just use uvx:
uvx tenzir 'version'
The bundled binary is available for Apple Silicon Macs, aarch64 Linux, and
x86_64 Linux. On other platforms, the wheel only contains the Python bindings
and you need to install the tenzir binary separately.
Dec 16, 2025 · @IyeOnline · #5597
We made an under-the-hood change to the publish and subscribe implementation
that reduces the overhead when publishing to high-throughput topics.
Dec 15, 2025 · @IyeOnline · #5593
We have removed the gcps:/ URI scheme, which previously would dispatch to
load_google_cloud_pubsub and save_google_cloud_pubsub. As these operators
are deprecated and will be removed, the schemas are being retired as well.
Tenzir now applies default retention policies for internal metrics and diagnostics:
tenzir.metrics.*): Retained for 16 days by defaulttenzir.diagnostics.*): Retained for 30 days by defaultThese defaults help manage storage usage while keeping sufficient history for troubleshooting. You can customize these settings:
# tenzir.yaml
tenzir:
retention:
metrics: 16d # Retention period for general metrics
diagnostics: 30d # Retention period for diagnostics
Set any retention period to 0 to disable automatic deletion for that category.
Dec 8, 2025 · @IyeOnline · #5595
When parsing typed-data (e.g. integers in JSON), with a predefined schema that
expected a different type (e.g. a time), the parser would crash with an
assertion failure.
This has now been resolved and the field will simply be null instead with a warning being emitted.
Dec 8, 2025 · @IyeOnline · #5445
Zeek JSON contains fields such as io.data.read.bytes and
io.data.read.bytes.per-second. These fields would previously overwrite each other
in order of appearance.
With this change bytes now is a record and the original value is kept under the
key "".
Dec 8, 2025 · @jachris · #5598
The non-actionable warning "received an operator metric without a unit" that was sometimes emitted for closed subpipelines was removed.
Dec 3, 2025 · @IyeOnline · #5590
We fixed a bug in a common component used across all parsers, which could enter an inconsistent state, leading to an "unexpected internal error: unreachable".
]]>Nov 28, 2025 · @IyeOnline · #5587
We fixed a bug in the back-pressure propagation across publish and subscribe,
that could cause pipelines to stall.
We addressed an issue that caused the python operator to fail when it received data containing timestamp values.
]]>All hash_* functions (MD5/SHA1/SHA2) now use OpenSSL's EVP implementation
instead of the previous digestpp copies. We've also added SHA3-224/256/384/512
variants to the hash_* family.
Here's how you compute a SHA3 hash digest:
from {x: hash_sha3_256("foo")}
{x: "76d3bc41c9f588f7fcd0d5bf4718f8f84b1c41b20882703100b9eb9413807c01"}
Nov 24, 2025 · @IyeOnline · #5582
In the last release we introduced a bug that made the write_parquet operator
exit early without writing the file. This has now been fixed.
Nov 24, 2025 · @IyeOnline · #5582
We addressed an assertion failure in the to_hive operator, which could fail
the partitioned writes for some data and format combinations.
Nov 19, 2025 · @IyeOnline · #5572
We have added two new functions: encode_base58 and decode_base58. As the names
imply, these Base58 encode/decode string or blob values, just like the already
existing functions for Base64, Hex or URL encoding.
With this change, we also enabled the usage of all encoding and decoding functions
on secret values.
Nov 19, 2025 · @jachris · #5580
Tenzir now supports the newly released OCSF version 1.7.0.
Nov 19, 2025 · @raxyte · #5571
The parse_time() function would earlier default to 1900 when the year was
unspecified. This has now been changed to 1970 to match the assumptions about
epoch in other parts of the language.
We now support multiple keys for deduplication, e.g., deduplicate a, b.c, d
is now equivalent to the previously unwieldy expression deduplicate {x: a, y: b.c, z: d}.
Nov 19, 2025 · @IyeOnline · #5579
We fixed a crash when performing some operations on data read from external parquet files, such as Amazon Security Lake.
Nov 19, 2025 · @jachris · #5568
Previously, the backpressure mechanism in publish and subscribe was not
working as intended. Thus, publishing pipelines continued processing data
even when downstream consumers were lagging far behind. This is now fixed. As a
result, memory consumption for pipelines connected by publish and subscribe
is reduced significantly in those cases.
We fixed a race condition in case multiple pipelines used the same transformation on the same data, such as string manipulations in where clauses to filter the input. The bug caused those pipelines to fail sporadically.
The node now prints a proper error message in case the builtin web server can't start because the port is already in use.
]]>Nov 24, 2025 · @IyeOnline · #5582
In the last release we introduced a bug that made the write_parquet operator
exit early without writing the file. This has now been fixed.
Nov 24, 2025 · @IyeOnline · #5582
We addressed an assertion failure in the to_hive operator, which could fail
the partitioned writes for some data and format combinations.
Nov 19, 2025 · @IyeOnline · #5579
We fixed a crash when performing some operations on data read from external parquet files, such as Amazon Security Lake.
]]>