Serving, Access Control, Batch Inference and Interfaces

What it is

Once a chain or agent is registered in Unity Catalog, Mosaic AI Model Serving runs it behind a REST endpoint. The LLM inside it is usually reached through Foundation Model APIs, and the endpoint needs credentials for every index, function or model it calls.

Not every workload is a live chat. Scoring a whole table belongs to ai_query() in SQL, and people reach the agent through an interface such as a Databricks App or a chat tool they already use.

How it works

Serving with Foundation Model APIs

Most Foundation Model API endpoints are OpenAI compatible, so the OpenAI client works against them. Your chain calls the LLM endpoint by name, and the agent itself is deployed to its own serving endpoint.

from databricks import agents
deployment = agents.deploy("main.docs.rag_chain", 3)
print(deployment.query_endpoint)

agents.deploy(model_name, version) creates the serving endpoint and also turns on MLflow tracing, inference tables, the Review App and production monitoring for the agent.

Controlling access to resources

The endpoint runs your code, so it needs an identity for each resource it touches.

With automatic passthrough, Databricks checks that the person deploying has access, for example Can Query on a serving endpoint, EXECUTE on a Unity Catalog function or SELECT on a table. Choose on behalf of user when each user must see only the data their own permissions allow. Choose manual authentication for resources outside Databricks, such as an external API key kept in a secret scope.

Who may call the endpoint itself is a separate setting: endpoint permissions are CAN VIEW, CAN QUERY and CAN MANAGE.

Batch inference with ai_query

ai_query() calls a model from SQL for every row, which suits scheduled jobs over tables: classifying tickets, summarizing reviews or extracting fields.

SELECT ticket_id,
  ai_query('support_batch_llm',
           CONCAT('Classify this ticket as billing, bug or other: ', body),
           failOnError => false) AS category
FROM main.support.tickets;

Databricks scales the batch for you, so submit the whole table in one query rather than splitting it. For production scale batches, use a batch optimized Databricks hosted model or a provisioned throughput endpoint; pay per token endpoints are meant for real time use. With failOnError => false the result is a struct holding the response and an error message, so failed rows are reported instead of stopping the job.

User facing interfaces

A Databricks App runs with its own service principal by default, or with user authorization when each user's permissions must apply. A Slack or Teams bot authenticates to the endpoint with a service principal.

How the exam asks about this

• "Nightly, summarize two million product reviews stored in a Delta table." Use ai_query() in a scheduled SQL job, not a real time endpoint loop.
• "The deployed agent returns permission errors when it calls the Vector Search index." Declare the index in resources so automatic authentication passthrough covers it.
• "A production chatbot needs guaranteed throughput on a fine tuned Llama model." Serve it with provisioned throughput.