Question 1

Finance says the workspace cost $94,000 last month and asks which team spent it. The usage dashboard does not break it down that way. Where do you actually look?

Accepted Answer

system.billing.usage is a Delta table you query with ordinary SQL, and custom tags on the workloads are what turn a DBU number into a team name. The usage dashboards are built on system tables, so they only slice the bill along the dimensions Databricks chose. system.billing.usage records every workload: which SKU ran, for how many DBUs, which workspace, and the tags attached to the cluster, warehouse, or job. Joining it to system.billing.list_prices on the SKU and the time window turns DBUs into an estimated dollar figure. Grouping by usage.custom_tags is the line that answers 'which team'. Without tags the table still tells you what ran, but not who owns it, and that is the gap the dashboard cannot close either. System tables are ordinary Delta tables in the system catalog. system.billing.usage records SKU, DBUs, workspace, and tags per workload. Join list_prices on SKU and the usage window to estimate dollars. custom_tags is the column that attributes spend to a team.

Question 2

You enabled system.billing yesterday and the CFO wants the last six months attributed. Why is that query going to disappoint them?

Accepted Answer

System schemas collect from the moment they are enabled. There is no backfill of history from before that. Each system schema has to be turned on per metastore, and several are off by default. Once enabled, rows start landing for new activity only. A query over system.billing.usage filtered to last quarter returns whatever has accumulated since you flipped the switch, which might be a day. The cloud provider's own billing export is the only record of the months before that, and it will not have your Databricks tags on it. The practical move is to enable every schema you expect to need on day one of a workspace, then tell stakeholders the attribution clock starts now. System schemas are enabled per metastore and several start off. Collection begins at enablement, with no backfill. Pre-enablement history only exists in the cloud bill, untagged. Enable the schemas you will need on day one of the workspace.

Question 3

Half the bill comes back with a null team tag. What produced those rows, and how do you stop it?

Accepted Answer

Anything created without a custom tag — an all-purpose cluster someone spun up, a warehouse, a job from the UI — lands as unattributed usage. usage.custom_tags only contains tags that were set on the compute or the job when it ran. A cluster created by clicking in the UI, a SQL warehouse with no tags, and a job defined before the tagging standard existed all bill to null. Serverless and system workloads such as predictive optimization bill under their own SKU and often have no team tag unless a budget policy assigns one. The fix is a default tag enforced where compute is created: a budget policy, a cluster policy that requires a team tag, and asset bundles that set the tag in YAML so a UI click is no longer the path. Then query the null-tag remainder weekly, because that query is the audit. Untagged clusters, warehouses, and jobs bill with a null team. Serverless and platform workloads need a budget policy to attribute. A cluster policy can require the tag before compute starts. The null-tag query is the ongoing audit, not a one-time cleanup.

Question 4

A SQL warehouse doubled in cost this week and nobody changed its size. Which system table tells you the query responsible?

Accepted Answer

system.query.history has the statement text, duration, and bytes scanned, so you can rank the warehouse's queries by what they actually cost to run. Warehouse size explains the rate, not the cause. system.query.history records each statement against a SQL warehouse: the text, the user, the warehouse id, total_duration_ms, and the bytes read. Sorting a single warehouse by duration or bytes over the week usually surfaces one dashboard query or a SELECT * that started scanning far more after a table grew. Cross-check the query profile for that statement to see whether it was a cache miss, a join explosion, or a full scan. Billing usage confirms the warehouse's DBU total; query history tells you which statement to fix. Filter on the start time or you will scan the whole history table. system.query.history stores statement text, user, duration, and bytes. Rank one warehouse's statements to find the one that grew. The query profile then explains why that statement is expensive. Filter on the time column or the history table itself is a large scan.

Question 5

Someone asks which downstream jobs break if you drop a column from a silver table. Which tables answer that?

Accepted Answer

system.access.table_lineage and system.access.column_lineage record reads and writes observed between tables, so you can walk what depends on that column. Unity Catalog records lineage as queries actually run. table_lineage connects a source table to a target table for each observed read, with the workspace, the user, and the time. column_lineage goes one level deeper and names the source and target columns, which is what you need for a drop-column question. A column with no downstream lineage for the last 90 days is a much safer drop than one feeding three gold tables. The limit is that lineage only exists for workloads that ran through Unity Catalog after the schema was enabled, so a job that has not run recently, or that reads the table from outside Databricks, will not appear. Lineage is recorded from observed reads, not from declared intent. column_lineage names the source and target columns of each read. A column with no recent downstream lineage is the safer drop. Jobs that have not run, or that read from outside UC, are invisible.

Question 6

Your cost query over system.billing.usage takes four minutes and scans tens of gigabytes. The table is not that conceptually big. What is going on?

Accepted Answer

System tables grow without your layout choices, and a query that does not filter the usage date scans the full retained history. You do not OPTIMIZE system tables and you do not choose their clustering. They are retained for a long window and appended continuously, so an unfiltered scan reads every workspace and every day the metastore has kept. usage_date and usage_start_time are the columns that prune. The same applies to system.access.audit and system.query.history, which get large faster than billing does. There is also ingestion lag: the last hour or two may not be there yet, so a query for 'right now' both scans a lot and still misses the newest rows. Materialise a daily aggregate of your own, filtered and tagged, if this query runs on a schedule. You do not control layout or retention of system tables. Filter on usage_date or the query reads the full retained history. audit and query.history grow faster than billing usage. Recent hours can be missing because ingestion lags.

Question 7

Security wants to know who granted a contractor access to a PII table last Tuesday. Which table, and what is the catch?

Accepted Answer

system.access.audit records the grant action, the principal who made it, and the object — if the audit schema was enabled before Tuesday. system.access.audit is the account's audit log as a table: logins, permission changes, share access, and job actions, with the user, the request, and the target. A GRANT shows up as an action name on the table's securable, with the identity that performed it. The catch is the same as billing: the schema must have been enabled at the time, and audit rows arrive with a delay. It is also a wide, chatty table, so filter on event_date and the action name before you scan it. For shared data, recipient reads are audited too, which is the record a file drop never gives you. system.access.audit records who did what, including grants. The schema has to have been enabled before the event. Filter on event date and action or the scan is huge. Recipient reads of a share are audited here as well.

Question 8

You are rolling Databricks out to twelve business units. What has to be true on day one so the bill is attributable in a year?

Accepted Answer

A required team tag enforced by cluster policy and budget policy, every system schema enabled, and a owned cost mart — because none of this can be reconstructed later. Attribution is a decision you make before the spend, not a query you write after it. Cluster policies and budget policies that reject untagged compute close the UI path. Asset bundles set the tag in code so jobs cannot drift. Enabling billing, access, query, and lakeflow schemas on each metastore starts the clock, and waiting a quarter means that quarter is unauditable. A small daily job that aggregates usage by tag into a table you own survives retention limits and gives finance a stable contract. The organisational part is harder than the SQL: someone has to own the tag taxonomy, and a tag that means three different things across units is as useless as a null. Policies must reject compute that has no team tag. System schemas start collecting only once enabled, with no backfill. Aggregate into a table you own so retention is your decision. A tag taxonomy with one owner is what makes the numbers mean anything.