Skip to main content
The repository includes a runnable virtual-environment project driven by synthetic e-commerce events:
  • Redpanda provides Kafka-compatible ingestion.
  • A local producer emits order lifecycle events.
  • ClickHouse stores managed sources, deployment tables, and stable views.
  • Ten SQL models exercise transforms, filters, aggregates, and one reference-style join.
This example enables [settings] virtual_environments = true because it demonstrates backfill, staged audit, and publish. For the default direct workflow, follow the quickstart.

​
Prerequisites

​
Get the project

From a StreamBuild source checkout:
The demo lives under examples/orders_demo/:

​
Start the stack

After both services report ready, confirm that the producer log contains tick: sent before continuing. The producer writes to source.order_events.live. Local endpoints are: The checked-in project target connects as clickhouse / clickhouse and uses the orders_demo database.

​
Inspect the graph

The compiled graph is:
orders and order_cancellations independently consume the managed source. Non-aggregate models preserve the four _replay_* lineage columns; aggregate models place replay predicates on their inputs instead of projecting post-aggregate lineage. orders feeds order_status_changes, order_items, region_lookup, and enriched_orders. order_status_changes feeds avg_fulfillment_time; order_items feeds daily_revenue and hourly_order_volume; and region_lookup is a side reference for enriched_orders. order_cancellations feeds daily_cancellation_rates.

​
Run SQL tests

The example covers model expectations, zero-row assertions, project macros, nested macro calls, and macro mode. Tests assemble their own CTE inputs, so they can run before the deployment graph exists.

​
Plan and backfill

Preview the initial deployment:
Create deployment-specific tables, attach live ingestion, capture watermarks, and replay retained Kafka history:
Review the displayed plan and approve it with y. The result prints the generated deployment ID and the exact audit and publish follow-up commands.

​
Audit the deployment

The assessment combines replay readiness with singular and generic SQL audits. The example checks status values, NULL identifiers, line totals, future events, and aggregate invariants. Stop and investigate if the assessment is not_ready; publish does not enforce that advisory result automatically.

​
Publish

Stable tbl__* views now point to the selected deployment tables. Query one through ClickHouse:
Run live audits after publish when you want a post-publish check:

​
Clean up

The -v flag removes Redpanda, ClickHouse, and log volumes for a clean next run.