Question 1

A teammate wants to classify a million support tickets by calling the model API from a Python loop over a collected DataFrame. What do you suggest instead?

Accepted Answer

ai_query is a SQL expression the engine batches and distributes, so the rows stay in the table and the calls go out in batches. collect() pulls the million rows to the driver, and a Python loop then sends them one at a time, with you owning retries, rate limits, and the write-back. ai_query('endpoint', text_col) is an ordinary expression in a SELECT, so it runs where the data already is. The engine groups rows into batches and sends those to the serving endpoint, which is both faster and simpler than the loop. The result comes back as a column you can filter, group, or write with CREATE TABLE AS. You still pay per call, but you stop paying the engineering cost of the loop. ai_query is a SQL expression, not a notebook loop. The engine batches rows and sends them to a serving endpoint. Rows stay distributed instead of being collected to the driver. The result is a normal column you can write with CREATE TABLE AS.

Question 2

You only need a sentiment label, not a free-form answer. Which function do you reach for, and why does the choice matter?

Accepted Answer

Use a task function such as ai_analyze_sentiment or ai_classify, because the result is a constrained value you can group by rather than prose you have to parse. ai_query with a prompt returns whatever the model writes, which changes shape between rows and between reruns. Task functions target one job: ai_classify picks from a label set you supply, ai_extract pulls fields into a structure, ai_summarize, ai_translate, ai_analyze_sentiment, ai_similarity, ai_forecast, and ai_parse_document each return a predictable type. A sentiment column that is always one of a few values can be grouped and joined. A paragraph cannot. When you do need ai_query, pass a response schema so the endpoint returns structured fields instead of text you split later. Task functions return a constrained type for one job. ai_classify, ai_extract, ai_summarize, and ai_forecast are SQL expressions. Free-form model text is hard to group, join, or test. ai_query with a response schema is the structured option.

Question 3

The classification query scanned 200 million rows and the invoice was painful. The WHERE clause looked selective. What is the most likely shape of the SQL?

Accepted Answer

The filter was applied to the model's output, so the endpoint was called for every row and only then were rows discarded. ai_query costs one inference per row that reaches it, and the engine does not know the model will reject most of them. WHERE ai_query(...) = 'urgent' evaluates the model for the whole table and filters afterwards. WHERE created_date >= ... AND region = 'EMEA' followed by ai_query in the SELECT list evaluates the model only for the rows that survive. The same applies inside a view: a downstream filter on the label does not push through the model call. Look at the query profile for the number of rows entering the AI operator. That number times the per-token price is the bill. Cost is one inference per row that reaches the function. A filter on the model's output runs after the calls. Filter on real columns first, then call the model. The query profile shows how many rows entered the AI operator.

Question 4

What is actually different between ai_query and a Python UDF that calls the same endpoint?

Accepted Answer

The engine batches and distributes ai_query, and the endpoint it calls is a governed Unity Catalog object. A UDF is opaque per-row code running in a Python worker. A Python UDF sends rows to a Python process, serializes each one, and runs your loop, and the optimizer cannot see inside it. ai_query is planned by the engine: it batches many rows into one request, runs those batches across the cluster, and retries against the serving endpoint. The endpoint is a Unity Catalog object, so grants decide who may call it, lineage records that the table was produced from it, and audit records the calls. AI Gateway can add rate limits, payload logging, and usage tracking in front of that endpoint. None of that exists for a URL hardcoded inside a UDF. The result is still one inference per row — batching changes throughput, not the bill's slope. ai_query is batched and planned; a Python UDF is opaque per-row code. The endpoint is a Unity Catalog object with grants and lineage. AI Gateway adds rate limits, logging, and usage tracking. Batching improves throughput and does not change the per-row price.

Question 5

The same ai_query call can hit a pay-per-token foundation model, a provisioned throughput endpoint, or a custom model you trained. What changes in the SQL, and what changes in the bill?

Accepted Answer

The SQL does not change — you name an endpoint. The bill changes from a per-token price to capacity you pay for whether or not queries arrive. ai_query takes an endpoint name, and any of those three can sit behind one. A foundation-model endpoint bills per token, so idle costs nothing and a big backfill costs a lot, all at once. Provisioned throughput reserves capacity, so latency is predictable and you pay for the reservation even on a quiet day. A custom model on Model Serving is your artifact, your latency, and your scale-to-zero or always-on choice. Picking provisioned throughput for a nightly batch that runs for twenty minutes means paying for capacity the other twenty-three hours and forty minutes. Picking pay-per-token for an interactive assistant means latency and rate limits you do not control. The SQL is the wrong place to encode that choice; the endpoint configuration is the right one. The SQL names an endpoint; it does not name a billing model. Pay-per-token is cheap when idle and expensive on a large backfill. Provisioned throughput is reserved capacity you pay for either way. A custom served model is the same SQL with your own latency and scale policy.

Question 6

You built a vector index yourself in a separate database and a second job embeds new rows into it. What does Vector Search with Delta Sync remove from that design?

Accepted Answer

The sync job. Delta Sync tracks the source Delta table through change data feed and updates the index as rows are inserted, changed, and deleted. A self-managed index is correct until the table changes, and then freshness is a pipeline you operate: detect changes, embed them, upsert them, and delete the ones that disappeared. In Delta Sync mode the index is defined on a Delta table column, Databricks can compute the embeddings, and change data feed drives incremental updates. Querying the index is a mix of vector similarity and keyword filters, so you can restrict by tenant or date as well as by meaning. The index is a Unity Catalog object, so grants decide who can query it. You can still supply your own vectors if you have an embedding model you do not want replaced. What you give up is control of the index internals. Delta Sync updates the index from the source table's change data feed. Inserts, updates, and deletes propagate without a job you maintain. Queries combine vector similarity with keyword filters. The index is a Unity Catalog object with its own grants.

Question 7

A materialized view calls ai_query and its refresh is fully recomputing every hour, and the labels change slightly each time. What two properties of the function cause that?

Accepted Answer

The output is non-deterministic, so incremental refresh cannot reuse prior results, and every refresh calls the model again for every row. An incremental refresh reuses rows whose inputs did not change. A non-deterministic expression has no stable output for a given input, so the planner cannot prove that yesterday's label is still valid and falls back to recomputing the view. Recomputing calls the endpoint for the whole table, every hour, which is the bill. The labels also drift: the same ticket can classify differently on Tuesday, so downstream aggregates move without any source data moving. Keep model inference in a table you append to, with the label stored as data, and let the materialized view aggregate that table. Inference becomes a deliberate write instead of a side effect of refresh. Non-deterministic expressions block incremental refresh. Each full refresh calls the model for every row again. Labels drift between refreshes even when the source did not. Store inference results as data and aggregate those.

Question 8

Who should be allowed to send production table data to a foundation-model endpoint, and what controls exist beyond a wiki page?

Accepted Answer

The endpoint and the index are Unity Catalog objects, so grants decide who calls them, and AI Gateway records and rate-limits what was sent. A URL in a notebook plus a personal token is invisible: anyone who can run the notebook can send any column anywhere, and you will not know. An endpoint registered in Unity Catalog is grantable, so only a specific group can invoke it, and lineage shows which tables were produced using it. AI Gateway in front of the endpoint adds rate limits, usage accounting, and payload logging, which is the record you need when someone asks what customer text left the account. Combine that with column masks so the PII columns are masked before they ever reach the prompt. The remaining decision is contractual: which provider, which region, and whether prompts are retained, and that one is not a Unity Catalog setting. A serving endpoint is a Unity Catalog object with grants. Lineage shows which tables were produced by which endpoint. AI Gateway rate-limits calls and can log payloads. Mask sensitive columns before they are interpolated into a prompt.