Driver OOM

collect() and toPandas() pull the cluster into one JVM. Executors stay green. The driver is the process that dies.

Lesson 28 of 29 · Spark path

Explain it at my level

Watch the canvas:reservedsession / metadataresult store / collect()Live simulation

Driver JVM heap (spark.driver.memory — 4g)

Reserved memory (300MB fixed)100%

Result store — collect() / broadcast / toPandas()0%

User objects, UI, metadata8%

0 MBHeap cap: 4096 MB

Data arriving at the driver:

Driver heap initialized. Reserved region locked.

Executors already finished. This JVM is the one collect() fills.