> ## Documentation Index
> Fetch the complete documentation index at: https://docs.streambuild.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Orders Demo

> Run a ten-model Kafka-to-ClickHouse project with tests, audits, replay, and publish.

The repository includes a runnable virtual-environment project driven by synthetic e-commerce
events:

* Redpanda provides Kafka-compatible ingestion.
* A local producer emits order lifecycle events.
* ClickHouse stores managed sources, deployment tables, and stable views.
* Ten SQL models exercise transforms, filters, aggregates, and one reference-style join.

<Note>
  This example enables `[settings] virtual_environments = true` because it demonstrates backfill,
  staged audit, and publish. For the default direct workflow, follow the
  [quickstart](/quickstart).
</Note>

## Prerequisites

* Python 3.12 or newer
* Git
* [`uv`](https://docs.astral.sh/uv/getting-started/installation/)
* [Docker with Compose](https://docs.docker.com/compose/install/)
* `curl`

## Get the project

From a StreamBuild source checkout:

```bash theme={null}
git clone https://github.com/chio-labs/streambuild.git
cd streambuild
uv tool install .
```

The demo lives under `examples/orders_demo/`:

```text theme={null}
examples/orders_demo/
  streambuild_project.toml
  sources/order_events.yml
  pipelines/order_events/
  macros/common.py
  tests/order_events/
  audits/
  producer/
  docker/compose.yml
```

## Start the stack

```bash theme={null}
docker compose -f examples/orders_demo/docker/compose.yml up -d --build

until docker compose -f examples/orders_demo/docker/compose.yml \
  exec -T redpanda rpk cluster health >/dev/null 2>&1; do
  sleep 1
done

until curl --fail --silent 'http://localhost:18123/ping' >/dev/null; do
  sleep 1
done

docker compose -f examples/orders_demo/docker/compose.yml logs --tail 1 producer
```

After both services report ready, confirm that the producer log contains `tick: sent` before
continuing. The producer writes to `source.order_events.live`. Local endpoints are:

| Service           | Address                  |
| ----------------- | ------------------------ |
| Redpanda broker   | `localhost:19092`        |
| Redpanda Console  | `http://localhost:18081` |
| ClickHouse HTTP   | `localhost:18123`        |
| ClickHouse native | `localhost:19000`        |

The checked-in project target connects as `clickhouse` / `clickhouse` and uses the `orders_demo`
database.

## Inspect the graph

```bash theme={null}
stb discover --project-dir examples/orders_demo
stb compile --project-dir examples/orders_demo
```

The compiled graph is:

```text theme={null}
source:order_events
  |-- orders
  |   |-- order_status_changes
  |   |   `-- avg_fulfillment_time
  |   |-- order_items
  |   |   |-- daily_revenue
  |   |   `-- hourly_order_volume
  |   |-- region_lookup
  |   `-- enriched_orders
  |       `-- ref:region_lookup
  `-- order_cancellations
      `-- daily_cancellation_rates
```

`orders` and `order_cancellations` independently consume the managed source. Non-aggregate models
preserve the four `_replay_*` lineage columns; aggregate models place replay predicates on their
inputs instead of projecting post-aggregate lineage.

`orders` feeds `order_status_changes`, `order_items`, `region_lookup`, and `enriched_orders`.
`order_status_changes` feeds `avg_fulfillment_time`; `order_items` feeds `daily_revenue` and
`hourly_order_volume`; and `region_lookup` is a side reference for `enriched_orders`.
`order_cancellations` feeds `daily_cancellation_rates`.

## Run SQL tests

```bash theme={null}
stb test --project-dir examples/orders_demo
```

The example covers model expectations, zero-row assertions, project macros, nested macro calls, and
macro mode. Tests assemble their own CTE inputs, so they can run before the deployment graph exists.

## Plan and backfill

Preview the initial deployment:

```bash theme={null}
stb plan --project-dir examples/orders_demo
```

Create deployment-specific tables, attach live ingestion, capture watermarks, and replay retained
Kafka history:

```bash theme={null}
stb backfill --project-dir examples/orders_demo
```

Review the displayed plan and approve it with `y`. The result prints the generated deployment ID
and the exact audit and publish follow-up commands.

## Audit the deployment

```bash theme={null}
DEPLOYMENT_ID="replace-with-id-from-backfill"
stb audit backfill \
  --project-dir examples/orders_demo \
  --deployment-id "$DEPLOYMENT_ID"
```

The assessment combines replay readiness with singular and generic SQL audits. The example checks
status values, NULL identifiers, line totals, future events, and aggregate invariants.

Stop and investigate if the assessment is `not_ready`; publish does not enforce that advisory
result automatically.

## Publish

```bash theme={null}
DEPLOYMENT_ID="replace-with-id-from-backfill"
stb publish \
  --project-dir examples/orders_demo \
  --deployment-id "$DEPLOYMENT_ID"
```

Stable `tbl__*` views now point to the selected deployment tables. Query one through ClickHouse:

```bash theme={null}
docker compose -f examples/orders_demo/docker/compose.yml exec clickhouse \
  clickhouse-client \
  --user clickhouse \
  --password clickhouse \
  --database orders_demo \
  --query "SELECT status, count() FROM tbl__orders GROUP BY status ORDER BY status"
```

Run live audits after publish when you want a post-publish check:

```bash theme={null}
stb audit --project-dir examples/orders_demo
```

## Clean up

```bash theme={null}
docker compose -f examples/orders_demo/docker/compose.yml down -v
```

The `-v` flag removes Redpanda, ClickHouse, and log volumes for a clean next run.