Databricks senior interview questions

Short: Not as table files. Risk is notebooks, logs, collect(), and who can run compute that reads the lake.

Detailed: Files stay in the data plane. Notebook outputs, driver pulls, and mis-shared workspaces can still leak samples. Private link, cluster policies, and no collect() on PII are the controls. Unity Catalog governs names and grants; it is not a second copy of the files.

Senior: Commands down, status up, files never leave the data plane. Circle the driver in the customer VPC.

Common mistake: Answering only 'data never leaves our VPC' and stopping.

Follow-up: What would you put on the architecture diagram in an interview?

Lesson · Simulation

Short: They are not reading through the Delta log — or the writer isn't Delta.

Detailed: Spark parquet. format on a Delta path, a broken symlink, or copying files in S3. DESCRIBE HISTORY should exist. If not, it's not a Delta table.

Senior: VACUUM deletes unreferenced files. Time travel older than retention dies. Never VACUUM 0 hours in prod.

Common mistake: Restarting the cluster to 'clear the partial write'.

Follow-up: When do you VACUUM and what's the footgun?

Lesson · Simulation

Short: Delta Sharing / UC shares — grants on a share, not a second lake.

Detailed: Recipient gets access through the sharing protocol. Your storage stays. Don't clone PII into their bucket 'to make BI easy'. Lineage should show the share.

Senior: The files. UC stores metadata and grants; the bytes stay in the owner’s storage unless you explicitly copy.

Common mistake: COPY INTO their account as the default share mechanism.

Follow-up: What must still live in the data plane?

Lesson · Simulation

Short: Land it in bronze, merge into silver on the business key, rebuild gold from silver — never patch gold by hand.

Detailed: If you edit gold, you can't reconstruct yesterday. MERGE in silver with event time. Gold is a projection.

Senior: Drop silver+gold, rebuild from bronze, numbers match. If they don't, gold had hidden logic.

Common mistake: UPDATE gold.daily_revenue in a notebook because finance asked.

Follow-up: What's the replay test?

Lesson · Simulation

Short: Isolate partitions / rows so commits don't conflict, or serialize writers.

Detailed: Two jobs rewriting the same files. Partition the table by the merge boundary (date). One writer per partition. Retry is OK; 50 retries is a design bug.

Senior: When you replace a whole partition from a correct snapshot — cheaper than matching every key.

Common mistake: Turning off optimistic concurrency.

Follow-up: When is a replaceWhere overwrite better than MERGE?

Lesson · Simulation

Short: Filters don't match clustered columns, or the reader bypassed Delta, or stats weren't used.

Detailed: Check PartitionFilters / data skipping in the scan. If you Z-ORDER columns you never filter, you paid a rewrite for nothing. Photon/warehouse still needs a selective predicate.

Senior: Better reads vs extra DBUs and longer MERGE (more rewrite). Measure scan bytes before/after, not just 'we optimized'.

Common mistake: Running OPTIMIZE daily on a table nobody filters.

Follow-up: What's the write-amplification tradeoff?

Lesson · Simulation

Short: Checkpoint, cloud event backlog, and whether files arrived in a different prefix.

Detailed: Don't reset the checkpoint as step 1 — you'll duplicate bronze. Compare source listing vs bronze _metadata.file_path. Replay a prefix with a bounded backfill stream.

Senior: Idempotent bronze (path as key) or a separate backfill table you merge once.

Common mistake: rm checkpoint and 'just rerun'.

Follow-up: How do you backfill without doubles?

Lesson · Simulation

Short: The bottleneck isn't a Photon-eligible operator — tiny files, skew, or a UDF/scan of the whole lake.

Detailed: Check Photon usage % in the SQL profile. If it's high and you're still slow, it's I/O or shuffle skew. If it's ~0%, find the fallback expression. Don't buy bigger Photon SKUs first.

Senior: Photon accelerates a good plan. It does not fix a bad model, tiny files, or collect().

Common mistake: Upgrading DBU SKU without a profile.

Follow-up: What's the interview closer?

Lesson · Simulation

Short: Look at all-purpose uptime, DBU SKU (Photon), and warehouses left in running.

Detailed: System tables / account console: idle clusters, SQL warehouses without auto-stop, Photon on workloads that don't benefit. Spark UI will look fine.

Senior: Auto-terminate all-purpose, job clusters only for Workflows, warehouses auto-stop, Photon where the plan is native.

Common mistake: Tuning spark.sql.shuffle.partitions to cut DBUs.

Follow-up: What policy would you add first?

Lesson · Simulation

Short: Expectations that drop rows, a wrong grain in silver, or a live table reading a stale source snapshot.

Detailed: Event log + UC lineage. Don't start in Spark UI Stages — DLT may have many datasets. Compare bronze counts vs gold. Check if development vs production mode used different data.

Senior: Full refresh on a sampled bronze, expect metrics, then prod. Never full-refresh prod as a test.

Common mistake: Scaling the DLT cluster first.

Follow-up: How do you test a DLT graph locally-ish?

Lesson · Simulation