Skip to main content

Marts ERD (data dictionary)

The dbt marts, the tables the API reads from the public schema, drawn as data-dictionary ERDs — one diagram per subject area, so each file is a normal size you can open in the IDE or in draw.io. Generated from the marts' schema.yml files: every column, its data_type and description, the primary key, and each foreign key constraint (within the area).

  • Entity view: the area's tables with headers (name, row count, grain) and key columns only, plus a line for every in-area foreign key.
  • Full view: the same layout with every column's type and meaning.
Subject areaEntityFull
Jurisdictionspng · drawiopng · drawio
Events and meetingspng · drawiopng · drawio
Bills and policypng · drawiopng · drawio
People and organizations (MDM)png · drawiopng · drawio
Electionspng · drawiopng · drawio
Nonprofits and grantspng · drawiopng · drawio
Topics and tagspng · drawiopng · drawio
Browse and reportspng · drawiopng · drawio
Otherpng · drawiopng · drawio

The PNGs carry the diagram inside them, so draw.io (or diagrams.net) can open a PNG and edit it. They are stored as normal git blobs (not LFS), so a checkout can preview them.

A foreign key that points at a table in another subject area is omitted from the area diagram (both ends must be in the same file). Pass --one-canvas only when you need every area on one canvas for tooling; that output is too large to open in most viewers.

How to read it​

  • Header colour: green is an event table (one row per event or record), orange an entity table (one row per thing, the target of other tables' foreign keys), and blue a summary table (one row per measure or group). Tables are named by the entity they hold, never dim_/fact_.
  • Column names: blue is the primary key, purple a foreign key, grey a column no key uses.
  • Lines: right-angled, from the foreign-key row to the key row it points at. A crow's foot means many, a bar means one.
  • Row counts: with --dsn the generator counts every mart that is built as a table in the gold schema (an exact count(*)) and shows N rows in its header. A mart that isn't built there says "rows not measured yet", never 0. The legend names the schema and the minute the counts were taken.
  • Subject-area totals: each area's title shows its table count, its row total and how many of its tables have a primary key. The total adds up only the tables that were measured, and the title says how many that was when some weren't. For a measured table the key is Postgres's own (pg_constraint), so a model that declares a key the database doesn't hold shows as missing. For an unmeasured one it is the dbt declaration.
  • Views in public: the API reads public, which holds views over gold. A view can't carry a primary key, so the keys are checked on the gold tables the views read.

Regenerate​

The drawer is the erd workspace package (packages/erd):

PYTHONPATH=packages/erd/src python -m erd --models dbt_project/models/marts --out web_docs/static/img/erd --name marts \
--dsn postgresql://postgres:password@localhost:5433/open_navigator --schema gold
node packages/erd/src/erd/drawio_render.js web_docs/static/img/erd/marts_*_entity.drawio web_docs/static/img/erd/marts_*_full.drawio

Leave out --dsn to draw from the dbt declarations alone (no counts). --dsn needs psycopg2 (pip install 'communityone-erd[db]'). Per-area routing is seconds, not minutes; row counts still take a couple of minutes on a large warehouse.

The second command needs Node 20.15 or later and playwright (npm install playwright). It loads draw.io's viewer from viewer.diagrams.net. Where that site is blocked, set VIEWER= to a local copy of viewer-static.min.js from the jgraph/drawio repository. Render is 1× by default (SCALE=2 for denser print). PNGs are capped at 3500 px on the longer side (MAX_SIDE=) so they open in the IDE; open the .drawio for full resolution.

Subject areas are set by name prefix in SUBJECT_AREAS (packages/erd/src/erd/dbt_model.py). A new mart lands in the first area whose prefix matches its name, or in "Other". Add a prefix there to place it.