spark/internals
advanced
Connecting…

Tungsten execution engine

Tungsten stores rows off-heap and generates JVM bytecode for whole stages so Spark SQL is not a naive iterator of JVM objects.

Explain it at my level

  1. Off-heap
  2. Codegen
  3. Cache-aware
Driver · SparkSession · DAG scheduler