The Green DAG That Lied

P2medium9 minIncident Response

It's 14:30. No red tasks. No failed runs. No SLA misses. But the churn model's precision has slid for a week, and someone finally traced it to a feature table that's been landing ~50% populated every single day. The pipeline swears it succeeded — nine days running. Retrying today's run changed nothing.

Missing feature rows / day
28,000 → 0 rows
Days of silent gap
9 → 0 days

The incident

A daily feature-export pipeline has reported success for 9 straight days while writing only about half the expected rows. The gap is stable, silent, and self-perpetuating — reruns succeed and fix nothing. A downstream model degraded before anyone noticed the data was incomplete.

Symptoms on the table

  • feature_export: 9 consecutive 'success', exit code 0, no retries
  • feature_table daily row count ~48-55% of the 90-day expected, every day
  • The shortfall is stable, not random — always roughly half
  • watermark table advanced past all 9 days as if they were complete
  • Churn model AUC down from 0.88 to 0.79 over the same window

Systems on the board

The real components in play for this incident — the surface you investigate when the clock starts.

source_extract
shard reader
feature_export
BashOperator
object store
s3://features/raw
feature_table
warehouse
watermark
progress marker
Airflow UI
all green

What you'll practice

This is a timed, hands-on incident in the Incident Response. You diagnose the symptom, trace it to a root cause across real components, and ship a fix before the clock runs out — the same loop you run on call, without the production blast radius.

Members-only challenge

Solve it in the Simulation Arcade.

The interactive workspace — live metrics, the component map, and the fix you ship — runs inside Petascale Labs. Sign in to start the clock.

Related topics

Browse the full Arcade

Every challenge maps to a stratum in the curriculum.