tycoon data analyze¶
Scaffold dbt staging models (and optionally Rill dashboards) for a registered source. The shortcut between "I just ingested data" and "I have a typed view of it / a dashboard."
Synopsis¶
tycoon data analyze SOURCE [OPTIONS]
Arguments:
SOURCE Name of a registered source
Options:
--no-dbt Don't generate dbt staging models
--rill Also generate Rill source / metrics_view / dashboard YAMLs
-h, --help Show this message and exit
What it does¶
analyze introspects the source's schema in data/raw.duckdb and generates two things:
- dbt staging models — one
stg_<source>__<table>.sqlper ingested table, plus aschema.ymldocumenting the columns. Lands underdbt_project/models/staging/. - (Optional) Rill artifacts — when
--rillis passed, also generates: rill/sources/stg_<source>__<table>.yaml(Rill source pointing at the Parquet export)rill/metrics/stg_<source>__<table>_mv.yaml(metrics view with auto-derived dimensions + one count measure)rill/dashboards/stg_<source>__<table>.yaml(explore dashboard)
Examples¶
# Just dbt staging models
tycoon data analyze github
# dbt + Rill in one shot
tycoon data analyze github --rill
# Skip dbt (Rill only) — useful when you already wrote staging models by hand
tycoon data analyze github --no-dbt --rill
Generated dbt staging structure¶
For source github with tables issues, pull_requests, commits:
dbt_project/models/staging/
├── stg_github__issues.sql
├── stg_github__pull_requests.sql
├── stg_github__commits.sql
└── schema.yml
Each stg_*.sql is a typed SELECT * from the raw schema, with _dlt_* internal columns excluded. Edit them to add typing, renaming, filtering — that's the whole point of staging.
schema.yml documents columns with their types pulled from DuckDB's information_schema. Run dbt docs generate (via tycoon data transform docs) to render the lineage.
Generated Rill structure¶
For each table, --rill writes a 3-file chain:
rill/sources/stg_github__issues.yaml # type: source, connector: local_file
rill/metrics/stg_github__issues_mv.yaml # type: metrics_view, model: stg_github__issues
rill/dashboards/stg_github__issues.yaml # type: explore, metrics_view: stg_github__issues_mv
The Parquet bridge is intentional — it needs no DuckLake extension, no catalog attach, and no live warehouse connection, so --rill output works anywhere Rill does. See Rill 0.86 architecture in the v0.1.3 release notes for how it was chosen.
Correction (2026-08-02). This note previously said the bridge existed because "SQLite-backed DuckLake catalogs hold an exclusive lock that breaks Rill-while-ingesting." That named the wrong backend. The lock belongs to the catalog's metadata database, not to DuckLake or to SQLite: a DuckDB-backed catalog (
ducklake:foo.ducklake, the default) is a DuckDB file, and DuckDB locks a database file to one process. A SQLite-backed catalog (ducklake:sqlite:foo.sqlite) is multi-process safe — Rill 0.86 was verified reading one live during ingest with zero lock conflicts. The bridge remains a reasonable default for the reasons above, but a DuckLake live connector is not blocked by locking. Evidence and full design in issue #71 and PR #74.
The auto-generated metrics view picks dimensions and measures heuristically:
- Dimensions: every
VARCHAR/BOOLcolumn - Measures: a default
count(*)plusavg()/sum()forDOUBLE/BIGINTcolumns
Customize the dashboards by hand-editing the YAML — analyze writes them, but won't overwrite an existing file. Re-running analyze skips files that already exist (with a warning) so your hand-edits are safe.
After analyze¶
Related¶
tycoon data sources run— populate the raw layer first- Reference: Templates — templates that come pre-analyzed
- Reference: Observability tables — auto-generated
_tycoon_*dashboards