Skip to main content
StreamBuild records normalized boundary columns so a model graph can be populated from retained stream history. Boundary mode belongs to each source declaration.

​
Boundary modes

The normalized columns are _replay_partition, _replay_offset, _replay_timestamp, _replay_landed_at, and _replay_cursor.

​
Inclusive, at-least-once cutoffs

Every replay boundary is inclusive. StreamBuild intentionally allows overlap between rows handled by an attached live materialized view and rows inserted by replay. This avoids missing data at the cutoff, but it is not an exactly-once guarantee. Model SQL and ClickHouse engines own deduplication or aggregate idempotency.

​
Replay anchors

A replay starts at a source or eligible upstream model and flows through materialized views into the selected closure. A model can be an anchor when it:
  • preserves the lineage required by its source mode
  • is non-aggregate
  • has no mutable side reference
  • does not set replay_anchor never
StreamBuild derives anchors from typed graph edges. A selected model also includes its descendants.

​
Direct mode

Direct builds replay retained history for the selected closure. There is no bounded change policy: the selected resources are rebuilt each time. Before teardown, StreamBuild verifies the source still covers any required historical range recorded by prior builds.

​
Virtual-environment replay

Virtual environments may seed older rows from the active table and replay only a newer range. Change policies use:
The same fields can appear in pipeline.yml or MODEL(); model wins over pipeline, which wins over project. Durations accept d, h, m, or s. When a model cannot preserve older seeded history, typically because aggregation prevents a clean split, bounded_replay_fallback controls the result: --full-refresh forces full replay for selected virtual-environment models. --start-time requests a bounded replay from an explicit time and requires active published roots.