DataYatriSpark, seen under the hood
Simulate

Simulation/Spark

Educational simulation
Spark

Caching & Persist

First action computes; second action hits the cache.

Watch the canvas:computed stagecached blockLive simulation
First action — count()scanread 1 TBshufflenetwork moveaggregatesum by regionPartitions discardedlineage is all that remains2nd count()would recompute everythingunpersist()give the memory back
Caching buys the second action a shortcut, using the first action's work.