Topics

Data engineering, by concept

272 concepts from across the stack - storage, ingestion, modeling, query engines, governance. Each one links to the courses that teach it, a free tool to practice it, and the related ideas to learn next.

Featured

★Slowly changing dimensions

All topics

A

access control ACID transactions adaptive query execution admission control aggregation pushdown airflow airflow operations airflow scheduler alerting anonymization apache arrow apache druid apache iceberg apache pinot apache spark autoscaling

B

backfills backpressure batch ingestion benchmarking bitemporal modeling bloom filters broadcast variables bus matrix

C

D

P

Q

quasi-identifiers query engine operations query engines query optimization query planning query profiling

R

ratio metrics RDDs re-identification real-time OLAP reconciliation redaction regex retries right to erasure risk scoring RocksDB row groups row-level deletes run-length encoding

S

T

table maintenance tiered storage time grain time spine time travel tokenization transaction log transformations and actions trino tungsten

U

unity catalog upserts

V

variant type vectorized execution volcano model

W

watermarks whole-stage codegen wide tables windowing write-ahead log

Z

z-ordering

Start with the fundamentals

The first chapter of every course is free to read - no account needed.

Browse the curriculum →Free tools

Featured

All topics

A

B

C

D

E

F

G

H

I

K

L

M

N

O

P

Q

R

S

T

U

V

W

Z

Start with the fundamentals