# Tenzir Node v4.6.0 ## 🚀 Features ### Add `export --internal` to access metrics Dec 1, 2023 · [@jachris](https://github.com/jachris) · [#3619](https://github.com/tenzir/tenzir/pull/3619) The new `--internal` flag for the `export` operators returns internal events collected by the system, for example pipeline metrics. ### Add `parse` operator Dec 1, 2023 · [@jachris](https://github.com/jachris) · [#3665](https://github.com/tenzir/tenzir/pull/3665) The new, experimental `parse` operator applies a parser to the string stored in a given field. ### Add `apply` operator Dec 1, 2023 · [@jachris](https://github.com/jachris) · [#3677](https://github.com/tenzir/tenzir/pull/3677) The new `apply` operator includes pipelines defined in other files. ### Implement context backends for the contextualizer Dec 1, 2023 · [@Dakostu](https://github.com/Dakostu) · [#3684](https://github.com/tenzir/tenzir/pull/3684) The closed-source `context` plugin offers a backend functionality for finding matches between data sets. The new `lookup-table` built-in is a hashtable-based contextualization algorithm that enriches events based on a unique value. The JSON format has a new `--arrays-of-objects` parameter that allows for parsing a JSON array of JSON objects into an event for each object. ### Implement a TCP loader Dec 1, 2023 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3664](https://github.com/tenzir/tenzir/pull/3664) We added a new `tcp` connector that allows reading raw bytes from TCP or TLS connections. ### Support comments in xsv parser Nov 29, 2023 · [@eliaskosunen](https://github.com/eliaskosunen) · [#3681](https://github.com/tenzir/tenzir/pull/3681) Use `--allow-comments` with the `xsv` parser (incl. `csv`, `tsv`, and `ssv`) to treat lines beginning with `'#'` as comments. ### Add file extension detection to `from`/`to` Nov 27, 2023 · [@eliaskosunen](https://github.com/eliaskosunen) · [#3653](https://github.com/tenzir/tenzir/pull/3653) When using `from ` and `to ` without specifying the format explicitly using a `read`/`write` argument, the default format is determined by the file extension for all loaders and savers, if possible. Previously, that was only done when using the `file` loader/saver. Additionally, if the file name would indicate some sort of compression (e.g. `.gz`), compression and decompression is performed automatically. For example, `from https://example.com/myfile.yml.gz` is expanded to `load https://example.com/myfile.yml.gz | decompress gzip | read yaml` automatically. ### Support `show`-ing all aspects at once Nov 26, 2023 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3650](https://github.com/tenzir/tenzir/pull/3650) Use `show` without an aspect to return information about all aspects of a node. ### Add ‘min\_events’ parameters to /serve endpoint Nov 24, 2023 · [@lava](https://github.com/lava) · [#3666](https://github.com/tenzir/tenzir/pull/3666) We optimized the behavior of the ‘serve’ operator to respond quicker and cause less system load for pipelines that take a long time to generate the first result. The new `min_events` parameter can be used to implement long-polling behavior for clients of `/serve`. ### Add `syslog` parser Nov 22, 2023 · [@eliaskosunen](https://github.com/eliaskosunen) · [#3645](https://github.com/tenzir/tenzir/pull/3645) The `syslog` parser allows reading both RFC 5424 and RFC 3164 syslog messages. ### Add `yield` operator Nov 21, 2023 · [@jachris](https://github.com/jachris) · [#3651](https://github.com/tenzir/tenzir/pull/3651) The new `yield` operator extracts nested records with the ability to unfold lists. ### Introduce an experimental `python` pipeline operator Nov 20, 2023 · [@lava](https://github.com/lava) · [#3592](https://github.com/tenzir/tenzir/pull/3592) The `python` operator adds the ability to perform arbitrary event to event transformations with the full power of Python 3. ### Add `from/load/to/save ` Nov 17, 2023 · [@eliaskosunen](https://github.com/eliaskosunen) · [#3608](https://github.com/tenzir/tenzir/pull/3608) The operators `from`, `to`, `load`, and `save` support using URLs and file paths directly as their argument. For example, `load https://example.com` means `load https https://example.com`, and `save local-file.json` means `save file local-file.json`. ## 🔧 Changes ### Implement a TCP loader Dec 1, 2023 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3664](https://github.com/tenzir/tenzir/pull/3664) We renamed the `--bind` option of the `zmq` connector to `--listen`. ### Rename pytenzir to tenzir Nov 29, 2023 · [@lava](https://github.com/lava) · [#3660](https://github.com/tenzir/tenzir/pull/3660) We renamed the name of our python package from `pytenzir` to `tenzir`. ### Consider discard, export, and import as internal operators Nov 21, 2023 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3658](https://github.com/tenzir/tenzir/pull/3658) Ingress and egress metrics for pipelines now indicate whether the pipeline sent/received events to/from outside of the node with a new `internal` flag. For example, when using the `export` operator, data is entering the pipeline from within the node, so its ingress is considered internal. ## 🐞 Bug Fixes ### Add `export --internal` to access metrics Dec 1, 2023 · [@jachris](https://github.com/jachris) · [#3619](https://github.com/tenzir/tenzir/pull/3619) `export --live` now respects a subsequent `where ` instead of silently discarding the filter expression. ### Support lists and null values and empty strings in XSV parser Nov 30, 2023 · [@dominiklohmann](https://github.com/dominiklohmann) · [#3687](https://github.com/tenzir/tenzir/pull/3687) The `csv`, `ssv`, and `tsv` parsers now correctly support empty strings, lists, and null values. The `tail` operator no longer hangs occasionally. ### Fix `sort` type check Nov 20, 2023 · [@jachris](https://github.com/jachris) · [#3655](https://github.com/tenzir/tenzir/pull/3655) Using the `sort` operator with polymorphic inputs no longer leads to a failing assertion under some circumstances. [ Download on GitHub ](https://github.com/tenzir/tenzir/releases/tag/v4.6.0) [Get the release artifacts and source code.](https://github.com/tenzir/tenzir/releases/tag/v4.6.0)