# Tenzir Node v4.9.0 ## 🚀 Features ### Implement a context content dumping mechanism Feb 21, 2024 · [@Dakostu](https://github.com/Dakostu) · [#3893](https://github.com/tenzir/tenzir/pull/3893) The new `context inspect ` command dumps a specific context’s user-provided data, usually the context’s content. ### Support parsing numeric timestamps since epoch Feb 20, 2024 · [@jachris](https://github.com/jachris) · [#3927](https://github.com/tenzir/tenzir/pull/3927) When specifying a schema with a field typed as `time #unit=`, numeric values will be interpreted as offsets from the epoch. ### Implement more malleable lookup data for contexts Feb 19, 2024 · [@Dakostu](https://github.com/Dakostu) · [#3920](https://github.com/tenzir/tenzir/pull/3920) The context match events now contain a new field `mode` that states the lookup mode of this particular match. The `enrich` operator gained a `--filter` option, which causes it to exclude enriched events that do not contain a context. ### Add running and paused times to pipeline metrics Feb 16, 2024 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3940](https://github.com/tenzir/tenzir/pull/3940) Operator metrics now separately track the time that an operator was paused or running in the `time_paused` and `time_running` values in addition to the wall-clock time in `time_total`. Throughput rates now exclude the paused time from their calculation. ### Implement `context load`, `context save`, and `context reset` Feb 13, 2024 · [@eliaskosunen](https://github.com/eliaskosunen) · [#3908](https://github.com/tenzir/tenzir/pull/3908) The `context reset` operator allows for clearing the state of a context. The `context save` and `context load` operators allow serializing and deserializing the state of a context to/from bytes. ### Add new ‘—file’ option to the python operator Feb 9, 2024 · [@lava](https://github.com/lava) · [#3901](https://github.com/tenzir/tenzir/pull/3901) The `python` operator gained a new `--file` flag that allows loading python code from a file instead of providing it as part of the pipeline definition. ### Handle nested fields and integers as selectors in JSON parser Feb 5, 2024 · [@jachris](https://github.com/jachris) · [#3900](https://github.com/tenzir/tenzir/pull/3900) The `--selector` option of the `json` parser now works with nested fields, and integer fields. ### Improve the `export` operator Feb 5, 2024 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3909](https://github.com/tenzir/tenzir/pull/3909) The `export` operator gained a `--low-priority` option, which causes it to interfere less with regular priority exports at the cost of potentially running slower. ### Update the main repository to include the pipeline run ID Feb 1, 2024 · [@Dakostu](https://github.com/Dakostu) · [#3883](https://github.com/tenzir/tenzir/pull/3883) Managed pipelines now contain a new `total_runs` parameter that counts all started runs. The new `run` field is available in the events delivered by the `metrics` and `diagnostics` operators. ### Add the `openapi` operator Feb 1, 2024 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3898](https://github.com/tenzir/tenzir/pull/3898) The `openapi` source operator generates Tenzir’s OpenAPI specification. Use `openapi | to ./openapi.yaml` to generate a file with the canonical format. ### Add structured\_data to syslog output Jan 31, 2024 · [@eliaskosunen](https://github.com/eliaskosunen) · [#3871](https://github.com/tenzir/tenzir/pull/3871) The `structured_data` field in RFC 5424-style syslog messages is now parsed and included in the output. ### Add `lines` printer Jan 31, 2024 · [@eliaskosunen](https://github.com/eliaskosunen) · [#3847](https://github.com/tenzir/tenzir/pull/3847) The `lines` printer enables simple line-delimited formatting of events. ### Add Bloom filter context Jan 31, 2024 · [@mavam](https://github.com/mavam) · [#3834](https://github.com/tenzir/tenzir/pull/3834) The new `bloom-filter` context represents large sets in a space-efficient manner. ### Update the main repository to include timestamped pipelines Jan 30, 2024 · [@Dakostu](https://github.com/Dakostu) · [#3869](https://github.com/tenzir/tenzir/pull/3869) `show pipelines` and the `/pipeline` API endpoints now include `created_at` and `last_modified` fields that track the pipeline’s creation and last manual modification time, respectively. Pipelines created with older versions of Tenzir will use the start time of the node as their creation time. ### Rewrite `chart` and `set-attributes` operators Jan 30, 2024 · [@eliaskosunen](https://github.com/eliaskosunen) · [#3866](https://github.com/tenzir/tenzir/pull/3866) The `chart` operator adds metadata to the schema of the input events, enabling rendering events as bar, area, line, or pie charts on app.tenzir.com. ## 🔧 Changes ### Context versioning Feb 21, 2024 · [@eliaskosunen](https://github.com/eliaskosunen) · [#3945](https://github.com/tenzir/tenzir/pull/3945) The binary format used by contexts for saving on disk on node shutdown is now versioned. A node can support loading of multiple different versions, and automigrate between them. ### Disable colors if `NO_COLOR` or not a terminal Feb 19, 2024 · [@jachris](https://github.com/jachris) · [#3952](https://github.com/tenzir/tenzir/pull/3952) Color escape codes are no longer emitted if `NO_COLOR` is set to a non-empty value, or when the output device is not a terminal. ### Remove reader, writer, and language plugin types Feb 1, 2024 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3899](https://github.com/tenzir/tenzir/pull/3899) We removed the `tenzir-ctl start` subcommand. Users should switch to the `tenzir-node` command instead, which accepts the same arguments and presents the same command-line interface. ### Allow plugins to bundle further builtins Jan 31, 2024 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3877](https://github.com/tenzir/tenzir/pull/3877) Plugins may now depend on other plugins. Plugins with unmet dependencies are automatically disabled. For example, the `lookup` and `enrich` plugins now depend on the `context` plugin. Run `show plugins` to see all available plugins and their dependencies. ### Replace `tenzir.db-directory` with `tenzir.state-directory` Jan 31, 2024 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3889](https://github.com/tenzir/tenzir/pull/3889) The option `tenzir.db-directory` is deprecated in favor of the `tenzir.state-directory` option and will be removed in the future. ## 🐞 Bug Fixes ### Fix restart on failure Feb 19, 2024 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3947](https://github.com/tenzir/tenzir/pull/3947) The option to automatically restart on failure did not correctly trigger for pipelines that failed an operator emitted an error diagnostic, a new mechanism for improved error messages introduced with Tenzir v4.8. Such pipelines now restart automatically as expected. ### Prevent duplicate fields in schema Feb 12, 2024 · [@jachris](https://github.com/jachris) · [#3929](https://github.com/tenzir/tenzir/pull/3929) Invalid schema definitions, where a record contains the same key multiple times, are now detected and rejected. ### Add support for commas in seconds in the time data parser Feb 8, 2024 · [@eliaskosunen](https://github.com/eliaskosunen) · [#3903](https://github.com/tenzir/tenzir/pull/3903) Commas are now allowed as subsecond separators in timestamps in TQL. Previously, only dots were allowed, but ISO 8601 allows for both. ### Fix logger deadlock in python tests Feb 7, 2024 · [@lava](https://github.com/lava) · [#3911](https://github.com/tenzir/tenzir/pull/3911) We fixed a rare deadlock by changing the internal logger behavior from blocking until the oldest messages were consumed to overwriting them. ### Update the repository to include lookup lifetime fixes Feb 6, 2024 · [@Dakostu](https://github.com/Dakostu) · [#3910](https://github.com/tenzir/tenzir/pull/3910) Retroactive lookups will now properly terminate when they have finished. ### Improve the `export` operator Feb 5, 2024 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3909](https://github.com/tenzir/tenzir/pull/3909) We fixed a bug that under rare circumstances led to an indefinite hang when using a high-volume source followed by a slow transformation and a fast sink. ### Gracefully handle misaligned header and values in `xsv` parser Feb 4, 2024 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3874](https://github.com/tenzir/tenzir/pull/3874) The `xsv` parser (and by extension the `csv`, `tsv`, and `ssv` parsers) skipped lines that had a mismatch between the number of values contained and the number of fields defined in the header. Instead, it now fills in `null` values for missing values and, if the new `--auto-expand` option is set, also adds new header fields for excess values. ### Make `/serve` more consistent Jan 31, 2024 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3885](https://github.com/tenzir/tenzir/pull/3885) The `/serve` API sometimes returned an empty string for the next continuation token instead of `null` when there are no further results to fetch. It now consistently returns `null`. [ Download on GitHub ](https://github.com/tenzir/tenzir/releases/tag/v4.9.0) [Get the release artifacts and source code.](https://github.com/tenzir/tenzir/releases/tag/v4.9.0)