Series
Career and Interviews
A roadmap for what to learn now, lessons from engineers already doing the work, and how to handle the data engineering system design interview.
3 posts
The Data Engineer Roadmap for 2026 (in an AI-Native World)
Every data engineer roadmap written before 2024 quietly assumed the bottleneck was writing the code. Learn Python and SQL, build a pipeline, ship it. That bottleneck is gone - AI writes most of it now, and writes it faster than you. So the 2026 roadmap can't be a checklist of tools; it has to be a map of the depth underneath them, the part that decides whether the AI-generated query is right and why the 3am pipeline failed. Here's that map, area by area, junior to senior, with where each layer actually bites - and where to practice it.
Read more →I Asked 100 Data Engineers What They Wish They Knew Earlier
I didn't run a formal survey - nobody handed out a clipboard. But if you read enough postmortems, sit in enough on-call channels, and watch enough 'I wish someone had told me this' threads go by, the regrets of a hundred data engineers start to rhyme. The surprising part isn't that people wish they'd learned more tools. It's that the same depth gaps recur, almost word for word: the model that lied and never errored, the retry that double-charged a customer, the two files with identical rows that cost ten times apart. Here are those recurring lessons, grouped into the themes they actually fall into - each with a free place to feel it.
Read more →How to Crack the Data Engineering System Design Interview
Most system design prep is built for backend engineers: design Twitter, design a URL shortener, talk about load balancers and read replicas. Then you walk into a data engineering loop and the prompt is design a near-real-time analytics pipeline, and none of the muscle memory transfers. Data system design turns on different hinges - data volume and velocity, batch versus stream, idempotency, backfills, schema evolution, partition skew, and freshness SLAs. The good news: there is a framework underneath it, and once you can run it live, every prompt starts to feel like the same five moves. This is that framework, with the exact questions to ask, the math to do out loud, and the follow-up questions interviewers use to find the edge of what you know...
Read more →