DataYatriSpark, seen under the hood
Simulate
spark/fundamentals
beginner
Connecting…

What happens when Spark executes a job?

Trace groupBy + show() from lazy code through the plan, job, stages, tasks, shuffle, and result — one picture.

Lesson 4 of 29 · Spark path

Explain it at my level

  1. Code
  2. Plan
  3. Job
  4. Stages
  5. Tasks
  6. Executors
  7. Shuffle
  8. Result
Watch the canvas:driver / planjobstagetaskpartitionshuffleresultLive simulation
DriverYour codegroupBy("region").sum("amount")Catalyst planwritten · zero I/Oshow()not called yetStagesStage 0 — map sideread partitions → shuffle write✂Stage 1 — reduce sidelocked — waiting for shuffle🔒TasksP0 → T0waitingP1 → T1waitingP2 → T2waitingP3 → T3waitingShuffleReducer 0waitingReducer 1waitingTransformations only — recorded, not executed.
Code → plan → job → stages → tasks → shuffle → result. One snippet, one picture.