Databricks senior interview questions

Short: Not as table files. Risk is notebooks, logs, collect(), and who can run compute that reads the lake.

Detailed: Files stay in the data plane. Notebook outputs, driver pulls, and mis-shared workspaces can still leak samples. Private link, cluster policies, and no collect() on PII are the controls. Unity Catalog governs names and grants; it is not a second copy of the files.

Senior: Commands down, status up, files never leave the data plane. Circle the driver in the customer VPC.

Common mistake: Answering only 'data never leaves our VPC' and stopping.

Follow-up: What would you put on the architecture diagram in an interview?

Lesson · Simulation

Short: They are not reading through the Delta log — or the writer isn't Delta.

Detailed: Spark parquet. format on a Delta path, a broken symlink, or copying files in S3. DESCRIBE HISTORY should exist. If not, it's not a Delta table.

Senior: VACUUM deletes unreferenced files. Time travel older than retention dies. Never VACUUM 0 hours in prod.

Common mistake: Restarting the cluster to 'clear the partial write'.

Follow-up: When do you VACUUM and what's the footgun?

Lesson · Simulation

Short: VACUUM physically deleted the data files those older versions point at, so the commits still exist but the Parquet does not.

Detailed: VACUUM removes files that the current version no longer references once they are older than delta.deletedFileRetentionDuration, and those unreferenced files are exactly what time travel reads. Commit history can outlive the data because delta.logRetentionDuration is a separate knob, so DESCRIBE HISTORY may still list a version you can no longer query.

Senior: Two independent dials. delta.deletedFileRetentionDuration decides how long VACUUM leaves unreferenced data files alone — restorability actually depends on that one. delta.logRetentionDuration decides how long the commit metadata survives. Set the file dial from the recovery requirement, accept that you pay storage for every superseded file for that long, and stop calling it a backup: a dropped table or a deleted storage prefix takes the log with it.

Common mistake: Saying 'time travel goes back 30 days' as if it were a fixed guarantee instead of a function of retention and VACUUM.

Follow-up: Which property do you raise if legal wants 60 days of restorability, and what does it cost?

Lesson · Simulation

Short: In-flight readers and the concurrent writer can fail on missing files, and every earlier version becomes unreadable — that retention floor exists to protect concurrent operations.

Detailed: Delta refuses a retention below the configured threshold unless someone sets spark.databricks.delta.retentionDurationCheck.enabled to false, precisely because a query that resolved its file list minutes ago may still be reading files VACUUM just deleted. Once it lands, only the current version is guaranteed readable, so RESTORE and VERSION AS OF for anything earlier are gone, along with change-feed and streaming replays that needed those files.

Senior: The retention check is a guardrail, not a nuisance — anyone disabling it should be able to name the longest-running reader on that table. I would VACUUM on a schedule with retention at or above the longest read plus the recovery window, keep it out of the write window on hot tables, and get real durability from something VACUUM cannot touch: cloud storage versioning and periodic DEEP CLONE snapshots to a separate location.

Common mistake: Disabling the retention check as a habit because the error message was in the way.

Follow-up: How do you get the storage savings safely instead?

Lesson · Simulation

Short: Delta Sharing / UC shares — grants on a share, not a second lake.

Detailed: Recipient gets access through the sharing protocol. Your storage stays. Don't clone PII into their bucket 'to make BI easy'. Lineage should show the share.

Senior: The files. UC stores metadata and grants; the bytes stay in the owner’s storage unless you explicitly copy.

Common mistake: COPY INTO their account as the default share mechanism.

Follow-up: What must still live in the data plane?

Lesson · Simulation

Short: Land it in bronze, merge into silver on the business key, rebuild gold from silver — never patch gold by hand.

Detailed: If you edit gold, you can't reconstruct yesterday. MERGE in silver with event time. Gold is a projection.

Senior: Drop silver+gold, rebuild from bronze, numbers match. If they don't, gold had hidden logic.

Common mistake: UPDATE gold.daily_revenue in a notebook because finance asked.

Follow-up: What's the replay test?

Lesson · Simulation

Short: Isolate partitions / rows so commits don't conflict, or serialize writers.

Detailed: Two jobs rewriting the same files. Partition the table by the merge boundary (date). One writer per partition. Retry is OK; 50 retries is a design bug.

Senior: When you replace a whole partition from a correct snapshot — cheaper than matching every key.

Common mistake: Turning off optimistic concurrency.

Follow-up: When is a replaceWhere overwrite better than MERGE?

Lesson · Simulation

Short: Filters don't match clustered columns, or the reader bypassed Delta, or stats weren't used.

Detailed: Check PartitionFilters / data skipping in the scan. If you Z-ORDER columns you never filter, you paid a rewrite for nothing. Photon/warehouse still needs a selective predicate.

Senior: Better reads vs extra DBUs and longer MERGE (more rewrite). Measure scan bytes before/after, not just 'we optimized'.

Common mistake: Running OPTIMIZE daily on a table nobody filters.

Follow-up: What's the write-amplification tradeoff?

Lesson · Simulation

Short: High-cardinality partition columns create a directory per value so every write drops tiny files; recreate the table with CLUSTER BY (customer_id, event_date).

Detailed: Hive partitioning makes the physical layout a function of cardinality, so a million customers means a million directories and a scan that launches a task per tiny file. Liquid clustering decouples key choice from directory structure: files are written at a normal target size and the keys only decide which rows sit together, with file-level min/max statistics doing the skipping.

Senior: Partitioning is a physical commitment; clustering is a layout the engine maintains for you. I would migrate with a CTAS into a CLUSTER BY table, cut readers over, then verify in the query profile — files pruned versus files read, and bytes scanned before and after. The real win is not that clustering is magic, it is that re-keying later costs an ALTER instead of a full rewrite, so the choice stops being permanent.

Common mistake: Adding more OPTIMIZE runs while keeping the partition column, so compaction fights the partition boundaries forever.

Follow-up: How do you prove skipping actually improved after the migration?

Lesson · Simulation

Short: It is the right default for the long tail with unknown or drifting access patterns; keep explicit keys where the predicate is known and under an SLA.

Detailed: CLUSTER BY AUTO lets Databricks choose and adjust clustering keys from observed query patterns and maintain them through predictive optimization, so you are not hand-auditing hundreds of tables. Explicit CLUSTER BY (cols) still wins where the access path is known and critical — a merge key you rely on for MERGE pruning — because you do not want keys drifting under a latency commitment.

Senior: Treat it as a portfolio: AUTO for the tables nobody can characterize, explicit keys for the handful with a contract — merge keys and headline dashboard filters. Then measure instead of assuming: sample query profiles for files-pruned ratios per table, and watch the optimization DBUs, because incremental clustering costs scale with churn rather than table size. That is the improvement over Z-ORDER's full rewrite, not a free lunch.

Common mistake: Turning AUTO on and assuming layout work is now free, with no maintenance running and no check that queries improved.

Follow-up: What would make you override AUTO with explicit keys on a specific table?

Lesson · Simulation

Short: Checkpoint, cloud event backlog, and whether files arrived in a different prefix.

Detailed: Don't reset the checkpoint as step 1 — you'll duplicate bronze. Compare source listing vs bronze _metadata.file_path. Replay a prefix with a bounded backfill stream.

Senior: Idempotent bronze (path as key) or a separate backfill table you merge once.

Common mistake: rm checkpoint and 'just rerun'.

Follow-up: How do you backfill without doubles?

Lesson · Simulation

Short: The bottleneck isn't a Photon-eligible operator — tiny files, skew, or a UDF/scan of the whole lake.

Detailed: Check Photon usage % in the SQL profile. If it's high and you're still slow, it's I/O or shuffle skew. If it's ~0%, find the fallback expression. Don't buy bigger Photon SKUs first.

Senior: Photon accelerates a good plan. It does not fix a bad model, tiny files, or collect().

Common mistake: Upgrading DBU SKU without a profile.

Follow-up: What's the interview closer?

Lesson · Simulation

Short: Look at all-purpose uptime, DBU SKU (Photon), and warehouses left in running.

Detailed: System tables / account console: idle clusters, SQL warehouses without auto-stop, Photon on workloads that don't benefit. Spark UI will look fine.

Senior: Auto-terminate all-purpose, job clusters only for Workflows, warehouses auto-stop, Photon where the plan is native.

Common mistake: Tuning spark.sql.shuffle.partitions to cut DBUs.

Follow-up: What policy would you add first?

Lesson · Simulation

Short: At uptime, not queries: auto-stop disabled, min clusters pinned above 1, a warehouse left running, or someone changed the size or type.

Detailed: Warehouses bill DBUs for the time they are up, so an idle warehouse with no auto-stop bills all night for zero queries. Chart warehouse DBUs from system.billing.usage next to query counts per warehouse, then check the warehouse settings for auto-stop, min clusters, size, and a Classic-to-Pro or Serverless switch.

Senior: Uptime × size × SKU is the bill; query count is not in that formula. Flat query volume with rising DBU hours means idle time, so the fix is auto-stop plus min clusters back to 1 — not a smaller size that makes every real query slower. Serverless changes the tradeoff because startup is seconds, so aggressive auto-stop stops being painful.

Common mistake: Opening the query profile of the slowest statement — the queries did not change, the running hours did.

Follow-up: What auto-stop value would you set for a 9-to-6 BI workload, and what breaks if it is too aggressive?

Lesson · Simulation

Short: The result cache was invalidated by the new table version and the warehouse's local disk cache is cold for the new files, so the first run pays a full read.

Detailed: Databricks serves byte-identical queries from a result cache, and that cache is invalidated when the underlying Delta table commits a new version. Below it, each warehouse cluster keeps a local SSD disk cache of Parquet it has already read, which is also empty after auto-stop, a restart, or a rewrite by OPTIMIZE — and if the load produced many small files, that cold read is expensive on top.

Senior: Three caches with different lifetimes: the result cache (per identical query, killed by a new table version), the disk cache on the cluster's local SSDs (killed by restart or auto-stop), and Spark's in-memory cache that BI never touches. I would warm it deliberately with a small scheduled job that runs the top dashboard queries right after the load commits, and keep gold compacted so the cold read is cheap when it does happen.

Common mistake: Blaming Photon or asking for a bigger warehouse when the first query after every load is simply uncached.

Follow-up: How would you make 9am fast without keeping the warehouse hot all night?

Lesson · Simulation

Short: Expectations that drop rows, a wrong grain in silver, or a live table reading a stale source snapshot.

Detailed: Event log + UC lineage. Don't start in Spark UI Stages — DLT may have many datasets. Compare bronze counts vs gold. Check if development vs production mode used different data.

Senior: Full refresh on a sampled bronze, expect metrics, then prod. Never full-refresh prod as a test.

Common mistake: Scaling the DLT cluster first.

Follow-up: How do you test a DLT graph locally-ish?

Lesson · Simulation

Short: All-purpose DBUs are a more expensive SKU than jobs compute, and you also pay for the cluster's idle hours; a job cluster exists for the run and terminates.

Detailed: In Databricks pricing, Jobs Compute is a cheaper DBU rate than All-Purpose Compute for the same instance type, and a job cluster starts per run and shuts down at the end so uptime matches work. Shared interactive clusters also leak state — installed libraries, cached data, someone else's runaway cell — into production runs, which is how a green pipeline fails only on Mondays.

Senior: There are two separate costs: the DBU rate and uptime you did not need. I would move scheduled work to job clusters or serverless, add a cluster policy so nobody can schedule production onto all-purpose, and keep interactive clusters for humans. If per-run startup latency is the genuine objection, serverless job compute answers it without the idle bill — the warm shared cluster answers it by paying for the cluster all night.

Common mistake: Arguing only about startup convenience and never comparing the DBU rate or the idle hours on the bill.

Follow-up: When is serverless job compute the better answer than a job cluster?

Lesson · Simulation

Short: Concurrent runs were allowed against a non-idempotent write, so two runs processed overlapping input and both appended.

Detailed: A job's max concurrent runs setting decides whether a scheduled run starts while the previous one is still going; at 1 the new run is skipped or queued instead of racing. The durable fix is making the write idempotent — MERGE on a business key, or a replaceWhere overwrite bounded to the run's window — so an overlap or a retry converges instead of duplicating.

Senior: Overlap is the symptom; the non-idempotent append is the bug. I would set max concurrent runs to 1 with a timeout and an alert so a slow run becomes visible rather than doubled, pass the window explicitly through job parameters and task values instead of each task calling now(), and make the sink a MERGE or a bounded replaceWhere so re-running a window is a no-op. Then define the job in Databricks Asset Bundles so concurrency, retries, and schedule are code, not something each workspace gets clicked differently.

Common mistake: Just lengthening the schedule interval, which hides the race until one run is slow again.

Follow-up: How would you pass the window boundaries between tasks so each task knows exactly what it processes?

Lesson · Simulation