dbt source freshness: configure checks and handle stale data
Configure dbt source freshness, choose warning and error thresholds, inspect results, and decide what downstream jobs should do when data is late.
Read article
Workflows tagged "data-pipelines"
Copyable approval workflows about data-pipelines.
Data reconciliation: a SQL example and an exception workflow
Compare source and target data with a runnable SQL example. Find missing rows and mismatched values, investigate differences, and review exceptions.
Read article
Data contracts: define expectations and review breaking changes
Define a data contract with schema, meaning, quality rules, and ownership. Separate automated validation from review of breaking changes.
Read article
Airflow backfill: rerun historical data safely
Plan an Airflow backfill, choose reprocessing behavior, limit concurrency, and review historical reruns before they change downstream data.
Read article
dbt full refresh: rebuild models with a reviewed scope
Use dbt full refresh for incremental models, check configuration overrides, preview model selection, and add approval before a production rebuild.
Read article