Simulation/Spark

Educational simulation
Spark

Partitions & Tasks

How a table splits into partitions and one task per partition.

Watch the canvas:partition (data)task (work)Live simulation
orders~1 TB across Parquet files on S3P0250 MBTask 0reads P0P1240 MBTask 1reads P1P2260 MBTask 2reads P2P3250 MBTask 3reads P310,000 files of 2 MB = 10,000 tasks that finish in 40 ms. Compact first.
Partition and task are the same unit seen from opposite sides.