spark/internals
advanced
Connecting…Catalyst optimizer
Catalyst rewrites your query four times. Exchange in the physical plan is the shuffle you will pay for.
Explain it at my level
- Unresolved
- Analyzed
- Optimized
- Physical
Step 1 of 4
- 1. Unresolved logicalunresolved attributes
- 2. Analyzed logicalcatalog + types bound
- 3. Optimized logicalpredicate pushdown, column prune
- 4. Physical planFileScan → Exchange → HashAggregate