spark/internals
intermediate
Connecting…The Spark DAG
The DAG is the graph of operators. Wide edges are shuffles. Reading the DAG is reading the cost.
Explain it at my level
- Nodes
- Edges
- Shuffle edge
Step 1 of 4
read parquet
→filter region
→groupBy region
→count() ⚡