Skip to content

v0.2.1

Released: 2026-09-16

Headline: multi-resource filesystem sources + validated config.

A filesystem source used to mean one path, one glob, one table. It can now declare several named resources and load them all in a single run, tycoon sources add walks you through them, and JSONL joins CSV and Parquet. Config that was previously guessed at is validated instead — which is the one change in this release that can break an existing project.

Almost everything here is on the ingestion side. The exception is tycoon city, which now lays the city out by pipeline depth, so a city rendered from the same catalog looks different from 0.2.0. Details under Landed and Upgrading from 0.2.0.

Landed

  • One source, many tables. A filesystem source can declare a resources: list, each entry with its own table_name, path and file_glob. A single tycoon data sources run loads all of them in one dlt pipeline run, each landing in its own table:
sources:
  arcade:
    type: filesystem
    schema: raw_arcade
    resources:
      - table_name: arcade_games
        path: data/input
        file_glob: games.csv
      - table_name: arcade_players
        path: data/input
        file_glob: players.csv

The older flat path/file_glob shape keeps working exactly as before. - tycoon sources add prompts for resources. Adding a filesystem source loops through resource entries instead of accepting a single path. A table name that isn't a valid identifier, or that repeats one you already entered, is refused with the reason and re-prompted — rather than written to tycoon.yml and failing later at load time. - JSONL files are parsed into rows. A *.jsonl glob pipes through dlt's read_jsonl(). Plain .json is deliberately excluded: a single JSON document isn't the newline-delimited shape read_jsonl() reads, and falling back with a warning beats parsing it wrongly. - A glob that matches nothing tells you. A typo'd glob and a source with genuinely nothing new to load used to look identical — both loaded zero rows in silence. Local globs that match no files now warn. Remote buckets (s3://, gs://, az://) are skipped rather than guessed at. - An unrecognized file_glob warns instead of quietly loading metadata. A glob that is neither CSV, Parquet nor JSONL still falls back to the raw filesystem resource, but now says it is loading file listings — path, size, modification time — not parsed rows. - tycoon data analyze --rill works on an already-scaffolded source. The already-referenced check used to skip the entire command, so analyze <source> --rill on a source that already had dbt staging models produced no dashboards at all — and the only way past it, --force, would have overwritten the existing dbt files. It now skips just the dbt scaffolding step. - The city is laid out by pipeline depth. tycoon city places gold and mart schemas downtown and sources on the periphery, so the shape of the city tells you where a table sits in your pipeline. Each schema is one neighbourhood on a street lattice; within a ring, the kerb nearest downtown goes to the schema feeding the most other schemas, then the larger one, then by name. Streets are drawn where routes and doors need them rather than everywhere, and every table now sits inside its own schema's district. Median route length on the dogfood catalog fell from 229 tiles to 36.

Upgrading from 0.2.0

  • Breaking: a filesystem source must declare path or bucket_url. With both missing, tycoon used to fall back to "." and silently scan the working directory. It now fails with an error naming the source. If a project relied on that default, make it explicit:
sources:
  files:
    type: filesystem
    config:
      path: data/input      # previously implicit
      file_glob: "*.csv"
  • Filesystem tables are named after the source, not after read_csv/read_parquet. Two filesystem sources sharing a schema used to collide into one table; each now lands in its own, named after the tycoon source. A project with data already in the old generic table gets a warning naming both tables on every run until the old one is gone. New rows only go to the new table, so any dbt model still selecting _read_csv or _read_parquet needs updating by hand, then the old table can be dropped.
  • schema_version stays at 2 — there is no migration to run.
  • The city looks different. The ring layout reorders the whole city, so a render of a catalog you looked at under 0.2.0 will not match your old screenshots. The city.json contract does not change: same top-level keys, same version: 1. Anything reading the export keeps working.
  • DATABASE_TYCOON_PLANNER does nothing now. It used to opt a run into the ring planner; that planner is the only one, so the variable is read by nothing. Leaving it set is harmless.
  • Projects scaffolded from the csv-import template before 0.2.1 carry a staging model that selects _read_csv. The template is fixed, but an existing scaffolded project needs the same by-hand update as above.

Known limitations

  • The legacy-table warning does not reach multi-resource sources. If you migrate a flat filesystem source to the resources: shape, the old generic table is left behind without the warning above — the one migration most likely to strand a dbt model is the one that stays quiet about it. Tracked for the next release.
  • A glob that matches nothing still truncates. The zero-match warning is warn-and-continue, and the load then runs with write_disposition="replace", so a previously loaded table is rewritten to zero rows while every other signal stays green. Tracked in #240, which also proposes surfacing it in the run ledger and an opt-in --fail-on-empty.
  • The city is a plan, not yet a simulation. The vehicle and guest simulations run but nothing is drawn, as in 0.2.0. The fix is open as #250 and lands in 0.2.2.
  • Coverage milestones are not surfaced yet. The exporter emits an achievements block the renderer still does not read, so a milestone shows as measured-zero rather than unknown. Fixed in #251, for 0.2.2.
  • The renderer's test suite is not CI-gated yet. That now includes the new planner's own tests under tests/tycoon_city/sim, which pass locally but are excluded from the default pytest run until #256 and #253 land the gate in 0.2.2.
  • Remote object storage is still unsupported. s3://, gs:// and az:// are recognized well enough to be skipped by the local-glob check, but are not a supported source yet.

For contributors

  • PR size and title checks run on every pull request, in warn-only mode: at most 8 counted files, a 500-line cap on newly added src/**/*.py files, and type(scope): description titles carrying a gh-<N> or PTC-<N> scope. Release-promotion PRs from a vX.Y.Z branch into main are exempt. Both checks read their config and scripts from the base ref, so a PR cannot raise its own limits.
  • A repository writing-style skill now documents the house style for documentation, GitHub issues, PR descriptions, release notes and review notes.

What's next

The rest of the city stream, already open as a stack of draft PRs (#248 through #256): the planner's tests joining the default pytest run, the vehicle, guest and achievements fixes that 0.2.0 listed as known limitations, and a CI gate for the web front end. After that, the layer-aware data CLI (fire, repair, firehouse, data layers, data health). The ingestion rewrite's M3 (#84) continues alongside.