Simulation/Spark

Educational simulation
Spark

Lazy Evaluation

Watch transformations pile up until an action finally runs.

Watch the canvas:plan node (nothing runs)action (job starts)Live simulation
read parquetFileScan nodefilter regionFilter nodegroupBy regionAggregate nodecount()action — job submittedDriver-side plan only. Cluster CPU used so far: none.Job 0 submitted. Now — and only now — executors open files.Because the filter existed before the scan, Parquet skips row groups.
Three of these four lines cost nothing. Only the last one starts a cluster.