tycoon data run-all¶
Ingest all registered sources, then run dbt build. The "rebuild everything" command.
Synopsis¶
tycoon data run-all [OPTIONS]
Options:
-n, --max-records INTEGER Cap records fetched per resource (useful for testing)
--skip-ingest Skip ingestion and only run dbt build
--skip-transform Skip dbt build and only run ingestion
-t, --target TEXT dbt target profile [default: dev]
--notify Send a webhook notification on completion (success/failure)
--fail-on-empty Fail when a local glob matches no files or a replace load brings in zero rows
-h, --help Show this message and exit
When to use it¶
Use cases:
- First-time setup — ingest + build in one command after
tycoon init. - Cron-driven refresh —
0 4 * * * tycoon data run-allfor a daily 4am rebuild. - CI smoke test — combined with
--max-records 10, exercises the full pipeline cheaply.
For a single source, prefer tycoon data sources run <name> + tycoon data transform run so you control what re-runs.
Behavior¶
For each source in tycoon.yml's sources: map:
- Runs the same ingest as
tycoon data sources run <name>(with--max-recordsif set). - If a source fails, the command stops there and exits 1. Sources after it don't run, and neither does
dbt build.
After every source has run:
- Runs
dbt buildagainst--target(skipped with--skip-transform).--skip-ingestskips step 1 and only builds.
With --fail-on-empty, a source whose local glob matches no files, or that loads zero rows with replace, counts as a failed source, so the command exits 1 before dbt build runs. See tycoon data sources run for the details.
If dbt build fails, the command exits non-zero. Any test failures from dbt build show up in tycoon data history show <invocation_id>.
Examples¶
# Full rebuild, abort on first error
tycoon data run-all
# Cheap cron-friendly refresh
tycoon data run-all --max-records 1000
# Just the ingest layer (skip dbt)
tycoon data run-all --skip-transform
# Just dbt build
tycoon data run-all --skip-ingest
What it doesn't do¶
- No Rill dashboard refresh as a separate step. That happens automatically as part of each ingest + dbt run via the observability hooks. After
run-all, dashboards are current. - No source ordering. Sources run in the order they appear in
tycoon.yml. There's no inter-source dependency model — that lives in dbt. - No retries, and no skipping a failed source. If a source fails, fix it and re-run.
- No partial-source selection.
run-allruns every source. Usetycoon data sources run <name>for targeted runs.
Observability¶
Every source ingest and the final dbt build are captured into .tycoon/metadata.duckdb exactly as if you'd run them individually. tycoon data history will show one entry per source plus one for the dbt build.
Notifications (--notify)¶
For unattended runs, --notify posts a webhook notification on completion: success when the run finishes, error (with the failing stage and a message tail) if ingestion or dbt build fails. Set $TYCOON_NOTIFY_WEBHOOK_URL first; which severities fire is governed by the notify.severities block in tycoon.yml. The webhook is best-effort — a notification failure warns but never fails the run. See tycoon notify for setup and payload shapes.
export TYCOON_NOTIFY_WEBHOOK_URL="https://hooks.slack.com/services/..."
tycoon data run-all --notify
Related¶
tycoon data sources run— single-source ingesttycoon data transform build— dbt build alonetycoon data history— review whatrun-alldidtycoon notify— the notification surface--notifyusestycoon schedule— runrun-all --notifyon a timer