Summary
Data publication methodology explains how source data becomes trusted operational state without blurring analytics, OLTP, and UI responsibilities. The method is built around rebuildable lakehouse layers and explicit publish steps.
Reader Question
How do bronze, silver, gold, serving layers, publish steps, snapshots, and freshness checks prevent analytics from leaking into OLTP or UI state?
Methodology
Ingestion captures external source data. Bronze preserves raw-ish source records. Silver canonicalizes entities and relationships. Gold expresses domain facts and analytics. Publish steps intentionally move selected state into serving tables or snapshots for UI and workflow consumption.
Jobs and DAGs orchestrate refresh order. Serving snapshots and quality checks make freshness visible. UI routes should read serving state or OLTP snapshots, not raw analytical joins.
Boundaries
Pipelines keep data correct and rebuildable. Jobs orchestrate refreshes. Services expose lakehouse and data-sync capabilities. Publish steps are the explicit boundary into operational serving state.
Tradeoffs
Publication discipline adds ceremony, but it makes future OLTP/analytics separation possible and keeps interactive surfaces from depending on mutable analytical internals.