Foundations of Context Engineering · Topic 3 of 3
Reasoning Modes and the Databricks Product Stack
Databricks Context Engineer: choose standard, extended or reduced thinking by token budget, and match Unity Catalog, Lakebase, MCP and MLflow 3 to needs.
- 5 min
- Unity Catalog governs and describes what reaches context; row filters and column masks follow the agent's identity.
- Lakebase is managed Postgres for low latency agent state and memory; Delta tables hold durable, auditable state.
- MCP connects tools; managed servers cover Unity Catalog functions, Genie spaces, Databricks AI Search and Databricks SQL, with no managed Files server.
What it is
Two kinds of choice sit under every Databricks agent design. The first is which product handles a need: Unity Catalog governs data and metadata, Lakebase stores fast agent state, MCP connects tools, and MLflow 3 traces and evaluates.
The second is how much the model should reason before it answers. The exam guide frames this as three reasoning modes: standard, extended thinking and reduced thinking, chosen by the task's difficulty, its latency and token budget, and its effect on the context window.
How it works
The product stack
| Product | Role in context engineering | Pick it when the scenario says |
|---|---|---|
| Unity Catalog | Governs what can reach context: tables, views, functions, volumes, comments, row filters, column masks, governed tags | "Only authorized data", "describe columns so the agent picks the right table", "mask PII" |
| Lakebase | Databricks managed Postgres (OLTP) database for low latency key lookups of agent state and memory | "Resume the session by thread ID", "store user preferences across sessions" |
| MCP | Standard protocol that exposes tools and data to agents; Databricks managed MCP servers exist for Unity Catalog functions, Genie spaces, Databricks AI Search and Databricks SQL | "Connect the agent to tools", "reuse the same tools across agents" |
| MLflow 3 | Tracing of every step, tool call and retrieved context; evaluation with mlflow.genai.evaluate() and scorers | "Find which step failed", "compare two configurations" |
Unity Catalog row filters and column masks apply to the identity the agent runs as, often a service principal. Table and column comments act as context: they help both agents and Genie choose and use the right data.
Lakebase and Delta tables are not interchangeable. Lakebase suits fast reads and writes of live state, while Delta tables suit durable, auditable task state that later runs and people read.
MCP has no managed Files server on Databricks. When a scenario needs a system no managed server covers, connect an external or custom MCP server.
Choosing a product: the deciding word
Distractors are usually a real product doing the wrong job. MLflow 3 records what happened but does not store session memory for the agent to read back, and Unity Catalog governs tools and data but does not run the agent's checkpoint store.
Reasoning modes
| Mode | Fits | Cost and context impact |
|---|---|---|
| Reduced thinking | Simple, high volume, latency sensitive steps: routing, classification, extraction, short lookups | Fewest reasoning tokens, lowest latency |
| Standard | Ordinary multi step tool use and question answering | Balanced default |
| Extended thinking | Hard planning, multi constraint reasoning, debugging, analysis where a wrong answer is costly | Many more reasoning tokens, higher latency and cost, more pressure on the window |
Reasoning tokens are not free. They add to the token budget and latency of each call, and in a long agent loop any reasoning carried forward in the history adds to context window pressure.
The justification the exam wants ties the mode to a stated requirement. "Strict sub second latency on a classifier" points to reduced thinking, while "accuracy on a complex multi step plan matters more than speed" points to extended thinking.
A worked budget
An agent handles 50,000 routing calls a day. If extended thinking adds about 1,500 reasoning tokens per call, that is 75 million extra tokens a day for a task a short answer already handles well, so reduced thinking is the better choice.
A quarterly planning agent runs ten times a day and fails when it skips constraints. Extended thinking adds only about 15,000 tokens a day there, and the accuracy gain justifies it.
Mixed systems often use both. A router in reduced thinking sends only the hard requests to a planner that uses extended thinking.
How the exam asks about this
- "The agent must resume a conversation after a restart using the thread ID with low latency." Choose Lakebase as the state store.
- "Find which tool call returned stale data during a failed run." Use MLflow 3 tracing.
- "A high volume intent classifier must stay under a strict latency target and a tight token budget." Choose reduced thinking.
What to remember for the exam
- Unity Catalog governs and describes what reaches context; row filters and column masks follow the agent's identity.
- Lakebase is managed Postgres for low latency agent state and memory; Delta tables hold durable, auditable state.
- MCP connects tools; managed servers cover Unity Catalog functions, Genie spaces, Databricks AI Search and Databricks SQL, with no managed Files server.
- MLflow 3 traces each step and evaluates configurations with scorers.
- Reduced thinking for simple, high volume, latency sensitive work; extended thinking for hard, high stakes reasoning; standard otherwise.
- Justify a reasoning mode with the stated requirement and its token and context cost.
Extended thinking is not a universal quality upgrade. On simple, high volume tasks it adds tokens and latency with little gain, and the exam marks it wrong when the scenario stresses budget or speed.
Common mistakes
Using MLflow as memory. Traces record what happened for debugging and evaluation. Session state the agent must read back belongs in Lakebase or a Delta table.
Forgetting the agent's identity. Unity Catalog permissions, row filters and column masks apply to whoever the agent runs as. If a service principal can see a column, so can the agent's context.
Picking extended thinking for every task. The right mode depends on difficulty and budget together. A router or classifier rarely needs long reasoning.
Check yourself
Which Databricks product stores per thread agent state for low latency resume?
Lakebase, the managed Postgres database, for example as a LangGraph checkpointer keyed by thread ID.
When is extended thinking justified?
When the task needs hard multi step reasoning and accuracy matters more than the extra tokens, latency and context window pressure.
An agent must call Genie spaces and Unity Catalog functions as tools. What connects them?
MCP, using the Databricks managed MCP servers for Genie spaces and Unity Catalog functions.
Topic 3 of 3 in Foundations of Context Engineering
Test yourself on Foundations of Context Engineering
The full Databricks Context Engineer Associate question bank has 600 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Databricks. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.