Home / Data Engineering

/data-engineering · TRACK

The Data Engineering track.

Kafka, Spark, Databricks, Airflow and dbt — pipelines that fail at 3 AM. Built as a curriculum — 5 chapters, recommended spine if you're new, side-quests when you're ready.

⏱ ~1h 31m total 4 beginner · 6 intermediate · 3 advanced 📚 13 topics

You'll cover: Kafka, Airflow, dbt — then go deeper.

Start here: Kafka Consumer Lag Growing but CPU Idle → Kafka Leader Not Available After Broker Restart → Kafka RecordTooLargeException →

Continue from Kafka Consumer Lag Growing but CPU Idle → Complete Data Engineering guide

Your progress 0 / 13 topics

0

Read

0

Chapters done

0h

Time invested

🔥 Start your streak today

CURRICULUM

5 chapters · 13 topics

01

Kafka

SPINEIntermediate

4 topics · ~28m · 0/4 read

Kafka Consumer Lag Growing but CPU Idle Kafka Leader Not Available After Broker Restart Kafka RecordTooLargeException Kafka Exactly-Once Semantics Broken by Producer Retries

02

Airflow

SPINEIntermediate

1 topics · ~7m · 0/1 read

Airflow Task Stuck in Queued Forever

03

dbt

SPINEBeginner

1 topics · ~7m · 0/1 read

dbt Compilation Error: Model Depends on a Node Not Found

04

Spark

Intermediate

5 topics · ~35m · 0/5 read

Spark Job Aborted: Task Failed 4 Times — Read the Real Cause Spark OutOfMemoryError in Executor During Shuffle Spark Data Skew: One Task Runs for Hours Spark AnalysisException: Cannot Resolve Column Name Spark Small Files Problem Destroys Read Performance

05

Databricks

Beginner

2 topics · ~14m · 0/2 read

Databricks Delta Concurrent Append Exception Databricks Cluster Terminated: Spot Instance Reclaimed

START HERE · CHAPTER 01

Kafka

4 topics · Begin with Kafka Consumer Lag Growing but CPU Idle

Start reading →