Backfill

Verb · Development

Definitions

  1. Re-processing historical data through a pipeline to fill gaps caused by bugs, schema changes, new features, or late-arriving data. Requires idempotent pipelines that can safely re-run without duplicating results. One of the most common and time-consuming data engineering tasks.

    In plain English: Re-running a data pipeline on old data to fill in missing or corrected information.

    Example: "The new enrichment field needs a backfill of the last 90 days — make sure the pipeline is idempotent before running it."

Related Terms