CoursesChallenges

Designing a Production Ingestion Platform

Architect an end-to-end ingestion platform: contracts, taxonomy, multi-tenancy, DLQ/replay, cost, cross-region.

The architect capstone for Ingestion & Transport — event-driven vs batch decisioning, schema registry and data contract strategy, topic taxonomy, multi-tenant ingest, DLQ/replay/retention policy, cost and capacity modeling, cross-region, and self-service ergonomics.

Advanced12 chapters· 4h 25m· in Ingestion & Transport

Course content

  1. 01The Ingestion Platform Charter: What You're BuildingFree
  2. 02Event-Driven vs Batch: The Decisioning Framework🔒
  3. 03Schema Registry & Data Contracts Strategy🔒
  4. 04Topic Taxonomy & Naming🔒
  5. 05Multi-Tenant Ingest🔒
  6. 06DLQ, Replay & Retention Policy🔒
  7. 07Cost & Capacity Modeling🔒
  8. 08Cross-Region & Disaster Recovery🔒
  9. 09Governance, Lineage & PII at the Ingest Edge🔒
  10. 10Self-Service & Platform Ergonomics🔒
  11. 11Reliability: SLOs, Backpressure, Graceful Degradation🔒
  12. 12Capstone: Architect TheWorldShop's Ingestion Platform🔒

Prerequisites

↗CDC at Scale: Patterns & Pitfalls↗Stream Processing Architecture: Kafka Streams vs Flink vs Spark↗Backfill Strategies for Lakehouses

Read the first chapter free

Start reading now — no account required for the free chapters.

Start: The Ingestion Platform Charter: What You're Building →More in Ingestion & Transport