Performs live filtering of the import feed using a context and translates context updates into historical queries.
context::lookup name:string, key=field|list, [into=field, yield=field, separate=bool, live=bool, retro=bool, parallel=int]Description
Section titled “Description”The lookup operator performs two actions simultaneously:
- Translate context updates into historical queries
- Filter all data newly ingested into the node against any context updates
These two operations combined offer unified matching, i.e., automated retro matching by turning context updates into historical queries, and live matching with a context on the import feed.
The diagram below illustrates how the operator works:
name: string
Section titled “name: string”The name of the context to lookup with.
key = field|list
Section titled “key = field|list”A field, or list of fields to use for matching.
into = field (optional)
Section titled “into = field (optional)”The name of the field in which to store the context’s enrichment.
Defaults to the name of the context.
separate = bool (optional)
Section titled “separate = bool (optional)”When multiple fields are provided, duplicate the event for every provided field and enrich them individually.
When using the option, the context moves from <into>.<field...> to
<into> in the resulting event.
live = bool (optional)
Section titled “live = bool (optional)”Enables live lookup for incoming events.
By default, both retro and live lookups are enabled. Specifying either retro=true
or live=true explicitly disables the other.
retro = bool (optional)
Section titled “retro = bool (optional)”Enables retrospective lookups for previously imported events. The lookup
operator will then apply a context after a context update.
By default, both retro and live lookups are enabled. Specifying either retro=true
or live=true explicitly disables the other.
yield = field (optional)
Section titled “yield = field (optional)”Provide a field into the context object to use as the context instead. If the
key does not exist within the context, a null value is used instead.
parallel = int (optional)
Section titled “parallel = int (optional)”The number of partitions to open in parallel for retrospective lookups. This
number directly correlates with memory usage and performance of the lookup
operator.
Defaults to 3.
Examples
Section titled “Examples”Apply context to incoming and historical events
Section titled “Apply context to incoming and historical events”Apply the context feodo to incoming suricata.flow events as well as retro-match
any historical event to context updates:
context::lookup "feodo", key=src_ipwhere @name == "suricata.flow"Apply context to incoming events
Section titled “Apply context to incoming events”Apply the context feodo to incoming suricata.flow events. In this example,
only events that are freshly imported will be matched against the fedo context.
context::lookup "feodo", key=src_ip, live=truewhere @name == "suricata.flow"Apply context to historical events
Section titled “Apply context to historical events”Apply the context feodo to historical suricata.flow events with every update
to feodo. In this example, on any context::update to the fedo context,
historical data persisted in the node will be retro-matched against the update.
context::lookup "feodo", key=src_ip, retro=truewhere @name == "suricata.flow"