The heap splits into Reserved, Unified, and User memory.
On the canvas
Three bars. A fixed 300 MB reserved sliver, one large Unified pool, and User memory for everything else.
In plain English
Spark carves out 300 MB it will never hand out, then takes spark.memory.fraction (0.6 by default) of the rest as the Unified pool. What is left is User memory: your Python or Scala objects, UDF state, and class metadata.
Without this step
Setting spark.executor.memory = 8g and assuming 8 GB is available for cache. Roughly 4.6 GB reaches the Unified pool, and that is before container overhead.
What lives in that big Unified pool?
