Driver OOM

collect() piles rows onto the driver. Executors stay green. The notebook dies.

Watch the canvas:reservedsession / metadataresult store / collect()Live simulation

Driver JVM heap (spark.driver.memory — 4g)

Reserved memory (300MB fixed)100%

Result store — collect() / broadcast / toPandas()0%

User objects, UI, metadata8%

0 MBHeap cap: 4096 MB

Data arriving at the driver:

Driver heap initialized. Reserved region locked.

Executors already finished. This JVM is the one collect() fills.