DataYatriSpark, seen under the hood
Simulate
spark/performance
intermediate
Connecting…

Broadcast joins

Replicate a small dimension to every executor and skip shuffling the fact table.

Lesson 25 of 29 · Spark path

Explain it at my level

  1. Small table
  2. Broadcast
  3. Local join
Watch the canvas:small table (broadcast)fact (stays)driver collectLive simulation
Drivercollects build sideregions 20 MBbroadcastorders 2 TBnever shuffledcopySmall table is the build side.
Broadcast = copy the small side. The 2 TB side does not move.