Write assets in Python. rivers resolves the graph, plans execution, and runs the scheduler as compiled Rust — so nothing in the control plane waits on a Python interpreter.
pip install rivers
import rivers as rs
@rs.Asset
def raw_data():
return {"users": 100, "events": 5000}
@rs.Asset
def summary(raw_data: dict):
return f"{raw_data['users']} users"
repo = rs.CodeRepository(assets=[raw_data, summary])
repo.materialize()rivers reads the type-annotated parameters and builds the dependency graph. No wiring step.
Most orchestrators run their scheduler, planner, and daemon as Python processes. rivers compiles all of them into one binary. Python only runs inside your own asset code.
| Component | Python orchestrators | rivers |
|---|---|---|
| Graph resolution | Python | Compiled Rust (petgraph) |
| Execution planner | Python | Compiled Rust |
| Partition mapping | Python | Compiled Rust |
| Daemon: schedules, sensors, automation | Python | Compiled Rust, no GIL held |
| Web UI server | Python plus a JavaScript bundle | Rust: axum, Leptos, WebAssembly |
| Kubernetes integration | Python launcher | Rust operator with CRDs |
| Python interpreter on the control plane | Yes, everywhere | None |
Plan times stay below a millisecond on graphs with thousands of nodes. See the full comparison
An asset is what your pipeline produces. A task is a step inside one. You wire neither — rivers reads the function parameters, builds the edges, and lineage comes for free.
import rivers as rs @rs.Assetdef sessions(events, users): return join(events, users)sessions goes hot and the downstream assets light up.No separate webserver and daemon to babysit. No Postgres to provision. No workspace.yaml, and no docker-compose file.
The binary you run on your laptop is the same one that ships to production through the Helm chart.
Getting started$ rivers dev pipelines.analyticsNot a feature list. These are the differences you feel in week three.
Most orchestrators run one global daemon for every team. One slow sensor stalls everyone's triggers. rivers runs a daemon inside each code location, so a bad tick in one team's repo cannot touch another's.
How isolation worksConcurrency pool slots are leases. When a pod is killed mid-step, the lease expires and the slot returns. A dead worker cannot hold your warehouse capacity hostage. One asset can hold several pools at once.
ConcurrencyWrite async def on an asset and rivers awaits it on the tokio event loop. No wrapper and no separate decorator. The parallel executor caps async fan-out separately from its subprocess workers.
Put time-window, static, and dynamic dimensions in the same partition definition, with no cap on dimension count. Cron accepts a seconds field, so partitions and schedules can tick below a minute.
PartitionsAnnotate the context with your model — AssetExecutionContext[MyConfig] — and the type is read from the annotation. No config DSL, no run-config dictionary, and real autocomplete on context.config.
Set OTEL_EXPORTER_OTLP_ENDPOINT and spans flow to your collector. Python logging and context.log bridge into the same subscriber, so per-step log capture needs no setup.
Each of these is one annotation in rivers.
@rs.Asset
def running_total(
self: rs.SelfDependency[int],
events: list,
) -> int:
prev = self.get_inner()
return (prev or 0) + len(events)An incremental asset loads its previous output through a typed parameter. There is no graph cycle, and no workaround.
@rs.Task
def double(x: int) -> int:
return x * 2
@rs.Asset.from_graph()
def doubled():
m = numbers().map(double)
return total(m.collect_stream())A graph asset is a sub-DAG of tasks, and each task is its own step. Fan out with .map(); a streaming collect starts downstream work on the first result instead of waiting for the slowest branch.
@rs.Asset
async def prices():
return await client.fetch("/prices")
@rs.Asset
def report(prices, inventory):
return {**prices, **inventory}Async assets run as event-loop tasks while sync assets run in subprocess workers, inside the same run.
Every screen below is Rust — Leptos server-side rendering with WebAssembly hydration, on axum. Run state, materializations, and automation ticks stream over server-sent events, so there is a live indicator instead of a refresh button.
Overview — Run health, success rate, and every asset group in one screen.
Registering a code location is one kubectl apply. The operator resolves the tag to an immutable digest, reconciles a Deployment and Service, and registers the endpoint for the UI to discover.
There is no workspace file to maintain, no "add code location" flow in the UI, and no agent process to register against. Delete the resource to remove it.
Install on KubernetesapiVersion: rivers.io/v1alpha1
kind: CodeLocation
metadata:
name: analytics
spec:
image: ghcr.io/acme/pipelines
tag: v0.2.0
module: pipelines.analyticsInstall the package, decorate a function, and materialize it.