CDC at Scale: Patterns & Pitfalls
Outbox pattern, ordering, schema evolution, DLQ/replay, exactly-once sinks, and GDPR erasure for CDC.
The hard half of CDC — the outbox pattern, dual-write hazards, schema evolution propagation, ordering and per-key guarantees, dead letter queues and replay, exactly-once into lakehouse sinks, and GDPR erasure propagation across a streaming pipeline.
Course content
- 01The Dual-Write Hazard, Revisited at ScaleFree
- 02The Outbox Pattern🔒
- 03Ordering & Per-Key Guarantees Across Topics🔒
- 04Schema Evolution Propagation Through the Pipeline🔒
- 05Dead Letter Queues & Replay🔒
- 06Exactly-Once Into Sinks (Iceberg/Delta/Warehouse)🔒
- 07Initial Loads & Incremental Snapshots at Scale (Signaling)🔒
- 08GDPR Erasure Propagation Through CDC🔒
- 09Monitoring CDC: Lag, Slot Growth, Connector Health🔒
- 10Failure Modes & Recovery (slot bloat, snapshot restarts)🔒
- 11Capstone: Design a Resilient CDC Pipeline for TheWorldShop🔒
Prerequisites
What to learn next
Read the first chapter free
Start reading now — no account required for the free chapters.