An Airflow backfill creates runs for a historical date range. You might need one after fixing a transformation, restoring an upstream feed, or adding a dataset that needs earlier records. Before starting, check what each run will read and write. Running yesterday's task again does not necessarily reproduce yesterday's result.
This guide uses the Airflow 3.3.1 documentation. Check your installed version before copying commands; Airflow 2 examples use a different backfill CLI.
Airflow backfill vs catchup vs clearing tasks
| Mechanism | When to use it |
|---|---|
| Backfill | You deliberately choose a historical window to process |
| Catchup | You want the scheduler to create missing scheduled intervals |
| Clear a task | You want an existing task instance to run again after investigating its failure |
A logical date identifies a run's place in the schedule. It is not simply the time someone pressed Run. Inspect the data interval and timetable before using that date in extraction logic. Airflow's DAG run documentation explains these distinctions.
For example, a task that queries "the last 24 hours" relative to the current clock can read today's records when invoked for a historical interval. Make the input window explicit before scheduling a month of reruns.
Start with a small backfill
After replacing the DAG name and dates, this command creates a backfill. It is an execution command, not a preview:
airflow backfill create --dag-id orders_daily \
--from-date 2026-08-01 \
--to-date 2026-08-03 \
--reprocess-behavior failed \
--max-active-runs 1Airflow offers three reprocessing choices. none leaves existing runs alone; failed permits another run for a failed date; completed also permits reprocessing completed runs. Missing runs can be created. An existing queued or running latest run prevents another run for that date. The backfill's concurrency setting is independent of the DAG's setting. See the backfill reference.
Airflow also provides a dry-run option to inspect candidate dates. Check the installed command's options with:
airflow backfill create --helpA date preview does not estimate warehouse cost or prove that repeated writes are safe. Existing run states can also change between review and execution.
Inspect the downstream effects
Choose one historical interval in a nonproduction environment and inspect the resulting records. Check whether the task replaces a partition, merges by a stable key, or appends another copy. Include downstream work such as exports and notifications in that review.
| Question | Evidence to collect |
|---|---|
| Will a rerun duplicate records? | Destination keys and write behavior |
| Does the job read the intended history? | Rendered input interval and query parameters |
| Will it send something twice? | External calls, notification tasks, and their retry behavior |
| Can the warehouse absorb the work? | Recent comparable run duration and measured consumption |
| How will a partial result be repaired? | A recovery procedure for the affected partitions |
Treat estimates as estimates. If a cost projection comes from a smaller run, record that assumption rather than presenting it as a guaranteed ceiling.
Add approval when the scope needs review
A routine correction in a development dataset may fit an automatic policy. A large production rerun that republishes customer data may need a person to approve its scope. Put that decision before the protected writes or before submitting the batch, depending on what the approval covers.
Use a request shaped around the operation:
DAG: orders_daily
Environment: production
Historical window: the reviewed start and end dates
Code version: the release being executed
Reprocessing choice: failed
Concurrency: one active backfill run
Affected destinations: list the tables and exports
Recovery owner: the person responsible for partial completion
Approval deadline: the end of the agreed execution windowAirflow already supports native human review through its HITL operators. Use that interface when it suits the reviewers. Check rejection and downstream trigger rules when designing the DAG; collecting a response and authorizing a write are separate steps.
ActionBox is another option when people answer requests from several systems in a shared inbox. It records the decision; your integration remains responsible for submitting and tracking the Airflow work. An approval inside each historical run could generate many requests. Decide explicitly whether you need one batch decision or separate decisions per interval.
Recover without starting a second batch blindly
If the submission response is lost, first inspect Airflow for the submitted work. Retrying the command without reconciliation can change the set of runs being considered. Save the submitted scope and returned identifiers in the system driving the operation.
After approval, recheck that the code version, destinations, and date window still match what was reviewed. If they changed, request a new decision. When execution finishes, record the data validation result as well as the scheduler status. A successful task does not by itself establish that the output is complete.
For rebuilding a dbt model rather than rerunning Airflow intervals, use the dbt full-refresh guide. For late upstream data, see dbt source freshness. To connect an external reviewer, follow the verified approval callback guide.