Where can I learn pipeline design?

Pipeline design is taught across 1 course on Petascale Labs, including "Building Pipelines That Don't Page You". The first chapter of every course is free to read, no account needed.

What concepts are related to pipeline design?

Closely related topics include backfills, data contracts, dead letter queue, idempotency, partitioning.

Topic

Pipeline design

Pipeline design shows up across 1 course in 1 layerof the data platform stack. Here's where it's taught, a free way to practice it, and what to learn next.

Where it's taught

Orchestration & Pipelines

Building Pipelines That Don't Page You

Idempotency as a design choice, backfill as a first-class API, small DAGs with sharp edges, contract tests at the source, partitioning for the reader, dead-letter queues for data, where to put state, and how to design for a one-engineer team. The build-side answer to every failure mode in this stratum.

8 ch · 1h 29m

1 free

Start learning pipeline design free

The first chapter of every course is free to read — no account needed.

Start: Building Pipelines That Don't Page You →All strata

Where it's taught

Related topics

Start learning pipeline design free