spark/fundamentals
intermediate
Connecting…RDDs: lineage and partitions
The original Spark API. An RDD is a partitioned collection with lineage so lost partitions can be recomputed.
Explain it at my level
- Partition
- Lineage
- Recompute
Step 1 of 4
orders table
orders · ~1 TB across parquet files