Skip to content

Backfills

BackfillResult

Returned by CodeRepository.backfill().

Attribute Type Description
backfill_id str Unique identifier for the backfill
num_partitions int Total number of partitions in the backfill
num_runs int Number of runs created
status str Current status ("in_progress", "completed", "failed", "canceled")
completed int Number of completed partitions
failed int Number of failed partitions
canceled int Number of canceled partitions
run_ids list[str] IDs of runs created by the backfill
is_dry_run bool Whether this was a dry-run preview
partition_keys list[PartitionKey] Partition keys included in the backfill

BackfillStatus

Returned by CodeRepository.get_backfill().

Attribute Type Description
backfill_id str Unique identifier for the backfill
status str Current status ("in_progress", "completed", "failed", "canceled")
total_partitions int Total number of partitions
completed_partitions int Number of completed partitions
failed_partitions int Number of failed partitions
canceled_partitions int Number of canceled partitions
run_ids list[str] IDs of runs created by the backfill
error str \| None Error message if the backfill failed
tags list[tuple[str, str]] Tags attached to every run launched by the backfill
launched_by LaunchedBy Origin of the backfill; manual launches from the authenticated UI carry the acting user
action str \| None The verb the child runs execute. None means materialize

BackfillStrategy

Controls how partition keys are grouped into runs during a backfill.

BackfillStrategy.multi_run()

BackfillStrategy.multi_run() -> BackfillStrategy.MultiRun

One run per partition key (default).

BackfillStrategy.single_run()

BackfillStrategy.single_run() -> BackfillStrategy.SingleRun

All partition keys in a single run.

BackfillStrategy.per_dimension()

BackfillStrategy.per_dimension(
    multi_run: list[str],
    single_run: list[str],
) -> BackfillStrategy.PerDimension

Per-dimension control for multi-dimensional partitions.

Parameter Type Description
multi_run list[str] Dimensions iterated across runs (at least one required)
single_run list[str] Dimensions batched within each run (at least one required)

BackfillStrategy.PerDimension attributes:

Attribute Type
multi_run list[str]
single_run list[str]

Warning

A dimension cannot appear in both multi_run and single_run.

PartitionKeyRange

Specifies a range of partition keys for a backfill.

PartitionKeyRange.single()

PartitionKeyRange.single(from_key: str, to_key: str) -> PartitionKeyRange

Range of single-dimension partition keys from from_key to to_key (inclusive).

rs.PartitionKeyRange.single("2024-01-01", "2024-01-31")

PartitionKeyRange.multi()

PartitionKeyRange.multi(
    dimensions: dict[str, tuple[str, str] | list[str]],
) -> PartitionKeyRange

Multi-dimension partition key range. Each dimension value can be:

  • A (from, to) tuple for a range
  • A list[str] for explicit keys
rs.PartitionKeyRange.multi({
    "date": ("2024-01-01", "2024-01-07"),
    "region": ["us", "eu", "asia"],
})

CodeRepository.backfill()

CodeRepository.backfill(
    selection: list[str] | None = None,
    partition_keys: list[PartitionKey] | None = None,
    partition_range: PartitionKeyRange | None = None,
    strategy: BackfillStrategy | None = None,
    failure_policy: str = "continue",
    max_concurrency: int = 4,
    tags: list[tuple[str, str]] | None = None,
    config: dict[str, Any] | None = None,
    block: bool = True,
    dry_run: bool = False,
    action: str | None = None,
) -> BackfillResult

Launch a backfill to reprocess partitions.

Parameter Type Default Description
selection list[str] \| None None Asset keys to backfill. None selects all partitioned assets — or, with action, every asset that defines the verb. With action, an empty list is an error.
partition_keys list[PartitionKey] \| None None Explicit list of partition keys to process.
partition_range PartitionKeyRange \| None None Range of partition keys. Mutually exclusive with partition_keys.
strategy BackfillStrategy \| None None How to group partitions into runs. Falls back to asset-level strategy, then MultiRun.
failure_policy str "continue" "continue" to keep processing on failure, "stop_on_failure" to halt.
max_concurrency int 4 Maximum number of concurrent runs.
tags list[tuple[str, str]] \| None None Tags attached to the backfill and its runs. Use ("rivers/priority", "N") to override default priority (-10).
config dict[str, Any] \| None None The launch document. Kept on the backfill record and applied to every child run.
block bool True If True, wait for the backfill to complete before returning. If False, only record the backfill; a daemon runs it later.
dry_run bool False If True, compute the plan without executing.
action str \| None None Run this action on every partition instead of materializing. Every selected asset must define the verb; child runs and rerun_backfill inherit it.

Note

Provide either partition_keys or partition_range, not both.

Priority

Backfill runs default to priority -10 (lower than regular runs at priority 0), ensuring scheduled and manually triggered runs are dequeued first when a run queue is configured. Override with tags=[("rivers/priority", "5")].

CodeRepository.get_backfill()

CodeRepository.get_backfill(backfill_id: str) -> BackfillStatus | None

Get the current status of a backfill by ID. Returns None if not found.

CodeRepository.cancel_backfill()

CodeRepository.cancel_backfill(backfill_id: str) -> bool

Cancel a running or requested backfill. Returns True if the in-process coordinator was signalled (the backfill has live state in this process); returns False and falls back to a storage-level cancel marker otherwise.

CodeRepository.rerun_backfill()

CodeRepository.rerun_backfill(
    backfill_id: str,
    block: bool = True,
    dry_run: bool = False,
) -> BackfillResult

Launch a previous backfill again as a new backfill, with the same selection, strategy, failure policy, concurrency, tags and verb. The rerun replays every partition of the original backfill, not only the failed or canceled ones. Keys that are no longer valid for the current definitions are dropped. For a backfill of a job, the rerun raises ExecutionError if the job now runs a different verb than the recorded one.

Warning

A rerun of a delete backfill deletes every partition again, including partitions materialized again after the first run. To retry only some partitions, call backfill() with those partition_keys and the original action.

Parameter Type Default Description
backfill_id str required ID of the backfill to rerun.
block bool True If True, wait for the rerun to complete before returning.
dry_run bool False If True, compute the plan without executing.