# Tenzir Node v4.8.0 ## 馃殌 Features ### Implement the `diagnostics` operator Jan 22, 2024 路 [@Dakostu](https://github.com/Dakostu) 路 [#3828](https://github.com/tenzir/tenzir/pull/3828) The new `diagnostics` operator provides information about diagnostics that a pipeline may encounter during its lifetime. ### Add support for Cisco Firepower syslog messages Jan 22, 2024 路 [@eliaskosunen](https://github.com/eliaskosunen) 路 [#3833](https://github.com/tenzir/tenzir/pull/3833) The RFC 3164 syslog parser now supports years in the message timestamp. ### Add `xsv --no-header` Jan 19, 2024 路 [@eliaskosunen](https://github.com/eliaskosunen) 路 [#3821](https://github.com/tenzir/tenzir/pull/3821) The `csv`, `tsv`, `ssv` and `xsv` printers now support not printing a header line with the `--no-header` option. ### Add savers for curl connectors Jan 16, 2024 路 [@mavam](https://github.com/mavam) 路 [#3539](https://github.com/tenzir/tenzir/pull/3539) The `http` and `https` loaders now also have savers to send data from a pipeline to a remote API. The `http` and `https` connectors have a new flag `--form` to submit the request body URL-encoded. This also changes the Content-Type header to `application/x-www-form-urlencoded`. ### Improve handling of open file descriptors Jan 15, 2024 路 [@lava](https://github.com/lava) 路 [#3784](https://github.com/tenzir/tenzir/pull/3784) On Linux systems, the process metrics now have an additional value `open_fds` showing the number of file descriptors opened by the node. ### Add node health metrics Jan 14, 2024 路 [@lava](https://github.com/lava) 路 [#3736](https://github.com/tenzir/tenzir/pull/3736) A Tenzir node will now automatically collect and store metrics about disk, cpu and memory usage of the host machine. ### Support concepts in more places Jan 12, 2024 路 [@dominiklohmann](https://github.com/dominiklohmann) 路 [#3812](https://github.com/tenzir/tenzir/pull/3812) Concepts are now supported in more places than just the `where` operator: All operators and concepts that reference fields in events now support them transparently. For example, it is not possible to enrich with a lookup table against all source IP addresses defined in the concept `net.src.ip`, or to group by destination ports across different schemas with the concept `net.dst.port`. ### Add time parser Jan 12, 2024 路 [@eliaskosunen](https://github.com/eliaskosunen) 路 [#3738](https://github.com/tenzir/tenzir/pull/3738) The `time` parser allows parsing datetimes and timestamps from arbitrary strings using a `strptime`-like format string. ### Fix `export --live` and introduce `metrics` Jan 11, 2024 路 [@dominiklohmann](https://github.com/dominiklohmann) 路 [#3790](https://github.com/tenzir/tenzir/pull/3790) The `metrics` operator returns internal metrics events generated in a Tenzir node. Use `metrics --live` to get a feed of metrics as they are being generated. ### Update the main repository to include the pipeline manager autostart changes Jan 11, 2024 路 [@Dakostu](https://github.com/Dakostu) 路 [#3785](https://github.com/tenzir/tenzir/pull/3785) Pipeline states in the `/pipeline` API will not change upon node shutdown anymore. When a node restarts afterwards, previously running pipelines will continue to run while paused pipelines will load in a stopped state. ### Allow setting the header for csv, tsv, and ssv manually Jan 9, 2024 路 [@dominiklohmann](https://github.com/dominiklohmann) 路 [#3778](https://github.com/tenzir/tenzir/pull/3778) The `csv`, `tsv`, `ssv` and `xsv` parsers now support setting the header line manually with the `--header` option. ### Add GELF parser and document Graylog integration Jan 3, 2024 路 [@mavam](https://github.com/mavam) 路 [#3768](https://github.com/tenzir/tenzir/pull/3768) The new `gelf` parser reads a stream of NULL-byte terminated messages in Graylog Extended Log Format (GELF). ### Implement `timeshift` and `delay` operators Dec 23, 2023 路 [@dominiklohmann](https://github.com/dominiklohmann) 路 [#3701](https://github.com/tenzir/tenzir/pull/3701) The new `timeshift` operator adjusts timestamps relative to a given start time, with an optional speedup. The new `delay` operator delays events relative to a given start time, with an optional speedup. ### Implement the `lookup` operator Dec 22, 2023 路 [@Dakostu](https://github.com/Dakostu) 路 [#3721](https://github.com/tenzir/tenzir/pull/3721) The new `lookup` operator performs live filtering of the import feed using a context, and translates context updates into historical queries. This effectively enables live and retro matching in a single operator. ## 馃敡 Changes ### Switch from JSON to MsgPack data transport Jan 9, 2024 路 [@mavam](https://github.com/mavam) 路 [#3770](https://github.com/tenzir/tenzir/pull/3770) The `fluent-bit` source operator no longer performs JSON conversion from Fluent Bit prior to processing an event. Instead, it directly processes the MsgPack data that Fluent Bit uses internally for more robust and quicker event delivery. ## 馃悶 Bug Fixes ### Print Zeek TSV metadata when schema changes Jan 21, 2024 路 [@dominiklohmann](https://github.com/dominiklohmann) 路 [#3836](https://github.com/tenzir/tenzir/pull/3836) The `zeek-tsv` printer incorrectly emitted metadata too frequently. It now only writes opening and closing tags when it encounters a new schema. ### Fail properly when transfer breaks Jan 21, 2024 路 [@mavam](https://github.com/mavam) 路 [#3842](https://github.com/tenzir/tenzir/pull/3842) Failing transfers using `http(s)` and `ftp(s)` connectors now properly return an error when the transfer broke. For example, `from http://does.not.exist` no longer returns silently a success. ### Display failing pipeline diagnostics in `/serve` Jan 18, 2024 路 [@dominiklohmann](https://github.com/dominiklohmann) 路 [#3788](https://github.com/tenzir/tenzir/pull/3788) The `/serve` API now displays why a pipeline became unavailable in an error case instead of showing a generic error message. This causes runtime errors in pipelines to show up in the Explorer on app.tenzir.com. ### Override lookup-table context entries for duplicate keys Jan 12, 2024 路 [@dominiklohmann](https://github.com/dominiklohmann) 路 [#3808](https://github.com/tenzir/tenzir/pull/3808) Updating entries of a `lookup-table` context now overrides values with duplicate keys instead of ignoring them. ### Fix `export --live` and introduce `metrics` Jan 11, 2024 路 [@dominiklohmann](https://github.com/dominiklohmann) 路 [#3790](https://github.com/tenzir/tenzir/pull/3790) `export --live` sometimes got stuck, failing to deliver events. This no longer happens. ### Optimize pipeline when using `/pipeline/launch` Jan 11, 2024 路 [@jachris](https://github.com/jachris) 路 [#3801](https://github.com/tenzir/tenzir/pull/3801) The `/pipeline/launch` endpoint now optimizes the pipeline before starting it. ### Fix blob parsing with padding Jan 11, 2024 路 [@jachris](https://github.com/jachris) 路 [#3765](https://github.com/tenzir/tenzir/pull/3765) When reading Base64-encoded JSON strings with the `blob` type, `=` padding is now accepted. ### Fix the `tenzir/tenzir:latest-slim` image Dec 21, 2023 路 [@dominiklohmann](https://github.com/dominiklohmann) 路 [#3764](https://github.com/tenzir/tenzir/pull/3764) The `tenzir/tenzir:latest-slim` Docker image now sets a default `TENZIR_CACHE_DIRECTORY` automatically. [ Download on GitHub ](https://github.com/tenzir/tenzir/releases/tag/v4.8.0) [Get the release artifacts and source code.](https://github.com/tenzir/tenzir/releases/tag/v4.8.0)