DataYatriSpark, seen under the hood
Simulate

Simulation/Spark

Educational simulation
Spark

RDD lineage

Partitions plus lineage. DataFrames sit on top and get Catalyst.

Watch the canvas:RDD partitionlineageDataFrame pathLive simulation
RDD lineageHadoopRDDfilesMapPartitionsfilterDataFrame / DatasetCatalyst · TungstenPartitions + lineage. That is an RDD.
RDDs are lineage + partitions. DataFrames sit on top and get the optimizer.