DataYatriSpark, seen under the hood

Home Lessons Playground Interview

No sign-in neededStart learningStart

Course library

Spark · Databricks · Delta

Data Skew Broadcast Join AQE

Databricks Architecture Delta Lake Unity Catalog

Beginner Senior Scenario Based

Spark Execution Simulator Shuffle Simulator Join Simulator Data Skew Simulator AQE Simulator

Your progress is saved automatically — no account needed.

Educational simulation

AQE Simulator

Runtime coalescing and skew handling.

Step 1 of 4

200 shuffle partitions (static)

STEP 1

Static plan

spark.sql.shuffle.partitions = 200 by default.

For a 1 GB shuffle, 200 partitions means 5 MB tasks — scheduler overhead. For 10 TB it may be too few.

DataYatri · no account needed

Lessons Playground Interview Sitemap