every

Runs a pipeline periodically at a fixed interval.

every interval:duration { … }

Description

The every operator repeats running a pipeline indefinitely at a fixed interval. The first run starts directly when the outer pipeline itself starts.

Every interval, the executor spawns a new sub-pipeline that runs to completion. If the sub-pipeline runs longer than interval, the next run starts immediately.

Examples

Produce one event per second and enumerate the result

every 1s {
  from {}
}
enumerate

{"#": 0} // immediately
{"#": 1} // after 1s
{"#": 2} // after 2s
{"#": 3} // after 3s
// … continues like this

Aggregate metrics periodically with `summarize`

Using every with summarize does not work as expected because summarize never terminates on its own — every would wait forever and the aggregation would never emit results.

Instead, use the options={frequency} parameter of summarize:

from_tcp "127.0.0.1:5432" { read_json }
summarize events=count(data), options={frequency: 5m}

Fetch the results from an API every 10 minutes

every 10min {
  from_http "example.org/api/threats" {
    read_json
  }
}
publish "threat-feed"