# Tenzir Node v5.11.0

This release delivers significant performance improvements for situations with many concurrent pipelines, making Tenzir more robust under high-load scenarios. New features include AWS role assumption support, enhanced string trimming functionality, and improved HTTP error handling capabilities. Additionally, this release adds several new operators and comes with various bug fixes.

## 🚀 Features

### Roles in `save_s3` and `to_amazon_security_lake`

Jul 31, 2025 · [@raxyte](https://github.com/raxyte), [@IyeOnline](https://github.com/IyeOnline) · [#5391](https://github.com/tenzir/tenzir/pull/5391)

We have added new options to assume a role to the `save_s3` and `to_amazon_security_lake` operators. You can specify an AWS `role` and the operator(s) will assume this role for authorization and optionally. Additionally you can specify an `external_id` to use alongside the role.

### More supported types in `read_parquet`

Jul 31, 2025 · [@IyeOnline](https://github.com/IyeOnline) · [#5373](https://github.com/tenzir/tenzir/pull/5373)

Tenzir’s does not support all types that Parquet supports. We have enabled the `read_parquet` operator to accept more types that are convertible to supported types. It will convert integer, floating point, and time types to the appropriate (wider) Tenzir type. For example, if your Parquet file contains a column of type `int32`, it will now be read in as `int64` instead of rejecting the entire file.

### Trimming custom characters

Jul 31, 2025 · [@mavam](https://github.com/mavam) · [#5389](https://github.com/tenzir/tenzir/pull/5389)

The `trim()`, `trim_start()`, and `trim_end()` functions can now remove specific characters from strings, not just whitespace. Pass a second argument containing a string where each character represents a character to remove:

```tql
from {
  path: "/path/to/file/".trim("/"),
  decorated: "--hello--world--".trim("-"),
  complex: "/-/data/-/".trim("/-")
}
```

```tql
{
  path: "path/to/file",
  decorated: "hello--world",
  complex: "data"
}
```

Each character in the second argument is treated individually, not as a complete string to match:

```tql
from {
  // Removes 'a', 'e', and 'g' from both ends
  chars: "abcdefg".trim("aeg"),
  // Removes any 'o', 'l', 'e', or 'h' from both ends
  word: "helloworldhello".trim("olleh")
}
```

```tql
{
  chars: "bcdf",
  word: "wr"
}
```

This also works with `trim_start()` and `trim_end()` for one-sided trimming:

```tql
from {
  start: "///api/v1/users".trim_start("/"),
  end: "data.csv.tmp.....".trim_end(".")
}
```

```tql
{
  start: "api/v1/users",
  end: "data.csv"
}
```

### Account key authentication for Azure Blob Storage

Jul 30, 2025 · [@raxyte](https://github.com/raxyte) · [#5380](https://github.com/tenzir/tenzir/pull/5380)

The `load_azure_blob_storage` and `save_azure_blob_storage` operators now support account key (shared key) authentication via a new `account_key` option. This provides an additional method for accessing Azure Blob Storage, alongside existing authentication options.

### New `read_all` operator

Jul 30, 2025 · [@jachris](https://github.com/jachris) · [#5368](https://github.com/tenzir/tenzir/pull/5368)

The `read_all` operator produces a single event for its entire input stream.

### Dynamic `log_type` for `to_google_secops`

Jul 29, 2025 · [@raxyte](https://github.com/raxyte) · [#5365](https://github.com/tenzir/tenzir/pull/5365)

The `to_google_secops` operator now supports dynamic `log_type`s. You can set the option to any expression evaluating to a string, e.g.:

```tql
from {type: "CUSTOM_DNS", text: "..."},
     {type: "BIND_DNS", text: "..."}
to_google_secops log_type=type, log_text=text, ...
```

### Versioned sources in `to_amazon_security_lake` operator

Jul 28, 2025 · [@IyeOnline](https://github.com/IyeOnline) · [#5369](https://github.com/tenzir/tenzir/pull/5369)

The `to_amazon_security_lake` operator now supports versioned custom sources, such as

```tql
let $lake_url = "s3://aws-security-data-lake-eu-west-2-lake-abcdefghijklmnopqrstuvwxyz1234/ext/tnz-ocsf-dns/1.0/"
to_amazon_security_lake $lake_url, …
```

### Dropping null fields

Jul 25, 2025 · [@mavam](https://github.com/mavam) · [#5370](https://github.com/tenzir/tenzir/pull/5370)

The new `drop_null_fields` operator removes fields containing null values from events. Without arguments, it drops all fields with null values. With field arguments, it drops the specified fields if they contain null values, and for record fields, it also recursively drops all null fields within them.

Drop all null fields:

```tql
from {
  id: 42,
  user: {name: "alice", email: null},
  status: null,
  tags: ["security", "audit"]
}
drop_null_fields
```

```tql
{
  id: 42,
  user: {
    name: "alice",
  },
  tags: [
    "security",
    "audit",
  ],
}
```

Drop specific null fields:

```tql
from {
  id: 42,
  user: {name: "alice", email: null},
  status: null,
  tags: ["security", "audit"]
}
drop_null_fields user.email
```

```tql
{
  id: 42,
  user: {
    name: "alice",
  },
  status: null,
  tags: [
    "security",
    "audit",
  ],
}
```

Note that `status` remains because it wasn’t specified in the field list.

When specifying a record field, all null fields within it are removed:

```tql
from {
  user: {name: "alice", email: null, role: null},
  settings: {theme: "dark", notifications: null}
}
drop_null_fields user
```

```tql
{
  user: {
    name: "alice",
  },
  settings: {
    theme: "dark",
    notifications: null,
  },
}
```

The `user.email` and `user.role` fields are removed because they are null fields within the specified `user` record. The `settings.notifications` field remains because `settings` was not specified.

### Handling HTTP error status codes

Jul 23, 2025 · [@raxyte](https://github.com/raxyte) · [#5358](https://github.com/tenzir/tenzir/pull/5358)

The `from_http` and `http` operators now provide an `error_field` option that lets you specify a field to receive the error response as a `blob`. When you set this option, the operators keep events with status codes outside the 200–399 range so you can handle them manually.

## 🔧 Changes

### Improvements to `context::enrich`

Jul 31, 2025 · [@jachris](https://github.com/jachris) · [#5388](https://github.com/tenzir/tenzir/pull/5388)

The `context::enrich` operator now allows using `mode="append"` even if the enrichment does not have the exact same type as the existing type, as long as they are compatible.

Furthermore, `mode="ocsf"` now returns `null` if no enrichment took place instead of a record with a `null` data field.

### Performance improvements

Jul 30, 2025 · [@jachris](https://github.com/jachris) · [#5382](https://github.com/tenzir/tenzir/pull/5382)

Tenzir can now handle significantly more concurrent pipelines without becoming unresponsive. These improvements make the system significantly more robust under high load, with response times remaining stable even with thousands of concurrent pipelines.

## 🐞 Bug Fixes

### Fixed crash in `read_parquet`

Jul 31, 2025 · [@IyeOnline](https://github.com/IyeOnline) · [#5373](https://github.com/tenzir/tenzir/pull/5373)

Tenzir and the `read_parquet` operator only support a subset of all Parquet types. Reading an unsupported Parquet file could previously crash Tenzir in some situations. This is now fixed and the operator instead raises an error.

### Formatting `ip` and `subnet` values in `to_amazon_security_lake`

Jul 31, 2025 · [@raxyte](https://github.com/raxyte) · [#5387](https://github.com/tenzir/tenzir/pull/5387)

The `to_amazon_security_lake` operator now correctly formats `ip` and `subnet` values as strings and formats timestamps using millisecond precision, similar to the Security Lake built-in sources.

### Return type of `map` for empty lists

Jul 30, 2025 · [@jachris](https://github.com/jachris) · [#5385](https://github.com/tenzir/tenzir/pull/5385)

Previously, the `map` function would return the input list when the input was empty, possibly producing type warnings downstream. It now correctly returns `list<null>` instead.

### Context operator metrics

Jul 29, 2025 · [@jachris](https://github.com/jachris) · [#5383](https://github.com/tenzir/tenzir/pull/5383)

The data flowing through the `context::` family of operators is no longer counted as actual ingress and egress.

### Fixed `to_amazon_security_lake` partitioning

Jul 28, 2025 · [@IyeOnline](https://github.com/IyeOnline) · [#5369](https://github.com/tenzir/tenzir/pull/5369)

The `to_amazon_security_lake` incorrectly partitioned as `…/accountID=…`. It now uses the correct `…/accountId=…`.

### Fixed secrets in `headers` argument in `from_http`

Jul 26, 2025 · [@IyeOnline](https://github.com/IyeOnline) · [#5376](https://github.com/tenzir/tenzir/pull/5376)

We fixed a crash when using a secret in the `headers` argument of the `from_http` operator.

### Fixed issue with table creation in `to_clickhouse`

Jul 24, 2025 · [@IyeOnline](https://github.com/IyeOnline) · [#5360](https://github.com/tenzir/tenzir/pull/5360)

Multiple `to_clickhouse` operators can now attempt to create the same ClickHouse table at the same time without an error.

[ Download on GitHub ](https://github.com/tenzir/tenzir/releases/tag/v5.11.0)

[Get the release artifacts and source code.](https://github.com/tenzir/tenzir/releases/tag/v5.11.0)