Foundations of Context Engineering · Topic 2 of 3
Attention Budget and Long Context Degradation
Databricks Context Engineer: keep agent context small with minimal tools and just-in-time retrieval, find attention drains, fix long session decline.
- 5 min
- Every token competes for attention; quality drops well before the hard limit.
- Proactive strategies: minimal tools, progressive disclosure, just in time retrieval, scoped tool results, a concise prompt.
- The attention drain is the large element the task rarely uses, often raw tool output or unused tool schemas.
What it is
A model has a limited attention budget: every token in the window competes for its focus. The more low-value tokens you load, the less reliably the model uses the tokens that matter, long before the hard context limit is reached.
Context engineering therefore works in two phases. Proactive strategies keep the window lean from the start, and reactive interventions such as compaction restore quality when a long session has already grown too large.
How it works
Proactive strategies
| Strategy | What it does | Pressure it removes |
|---|---|---|
| Minimal tool set | Give the agent only the tools its task needs | Tool definitions loaded on every call |
| Progressive disclosure | Show tool names with short summaries; load a full schema only when the agent selects that tool | Large schemas for tools never used |
| Just in time retrieval | The agent calls a retrieval tool mid run when a step needs it | Chunks loaded up front that no step uses |
| Tool result scoping | Tools return only the needed rows or fields, a summary or a count | Raw dumps of tables, logs or JSON |
| Concise system prompt | State rules once, move rarely used detail elsewhere | Fixed tokens paid on every turn |
These act before compaction is needed. A question that asks how to reduce pressure "before compaction becomes necessary" wants one of these, not a summarization step.
Pre-inference retrieval still has its place: it is simple and fast when almost every request needs the same kind of chunk. Just in time retrieval wins when only some steps need knowledge, or the step itself decides what to look up.
Finding the attention drain
Start from the trace and list what fills the window on a typical call. Small worked example for a 128,000-token window:
| Element | Tokens | Used by the task? |
|---|---|---|
| System prompt | 3,000 | Yes |
| 45 tool schemas | 27,000 | 4 tools used |
| 20 retrieved chunks | 16,000 | 3 chunks cited |
| Raw SQL result of 2,000 rows | 40,000 | One total needed |
| Conversation history | 12,000 | Yes |
The raw SQL result and the 41 unused tool schemas take 40,000 plus about 24,600 tokens, roughly two thirds of the 98,000 used. The change most likely to improve focus is to scope the tool result to the aggregate the agent needs, then trim the tool set.
The disproportionate element is the one that is large and rarely used. A large element that every step needs, such as the core instructions, is not the problem.
Spotting long context degradation
Models recall facts in the middle of a long context less reliably than facts at the start or end. As a session grows, earlier tool results and constraints drift into that middle zone and quality falls even though the hard limit is far away.
You find the point empirically. Log MLflow 3 traces and run scorers such as correctness or groundedness, then compare scores against context size or turn number. The degradation point is where scores fall and errors such as mis-cited earlier tool results or ignored earlier constraints start to appear.
Interventions that restore performance
| Intervention | Keeps earlier facts? | Note |
|---|---|---|
| Compaction: summarize history, continue with summary plus recent items | Yes, in summary form | The usual answer when the session must continue unbroken |
| Clear consumed tool results | Yes, if the agent already used them | No model call; cheap |
| Offload notes or state outside the window | Yes, retrievable later | Fits long tasks with durable facts |
| Sliding window truncation | No, oldest turns are lost | Drops constraints stated early |
| Larger context window | Yes | Does not fix attention dilution |
| Disable retrieval after N turns | Yes, but no new lookups | Removes capability |
How the exam asks about this
- "Each call loads 60 tool schemas but the task uses five." Use a minimal tool set or progressive disclosure so only needed schemas enter the window.
- "A tool returns 5,000 raw log lines and the agent needs the error count." Scope the tool result to the count or a short summary.
- "Groundedness falls after turn 40 of a planning session; the agent must keep working in the same session and keep every earlier fact." Enable compaction with a summary plus recent items.
What to remember for the exam
- Every token competes for attention; quality drops well before the hard limit.
- Proactive strategies: minimal tools, progressive disclosure, just in time retrieval, scoped tool results, a concise prompt.
- The attention drain is the large element the task rarely uses, often raw tool output or unused tool schemas.
- Information in the middle of a long context is recalled less reliably than at the start or end.
- Find the degradation point by plotting MLflow 3 scorer results against context size or turn number.
- Compaction restores quality without losing facts; truncation and disabling retrieval lose facts or capability.
A larger context window is a trap answer for degradation. It raises the limit but leaves the window just as diluted, so the agent still misses facts buried in the middle.
Common mistakes
Reaching for compaction first. If the design question says "before compaction becomes necessary", the answer is a proactive strategy. Compaction is the reactive step once a long session has already grown.
Cutting the wrong element. Trimming the system prompt saves little if a raw tool dump is ten times larger. Measure each element's tokens and its use before you cut.
Assuming quality holds until the limit. Degradation starts gradually and is visible in scorer results. Waiting for a context overflow error means the agent has already been answering badly for many turns.
Check yourself
Why does progressive disclosure reduce context pressure?
Only short tool summaries are loaded at first, and a full schema enters the window only when the agent selects that tool.
An agent loads 20 chunks up front but most steps use none. Which strategy fits?
Just in time retrieval, so the agent calls a retrieval tool only on the steps that need knowledge.
How do you identify where long context starts to hurt quality?
Run MLflow 3 scorers over traces and compare the scores against context size or turn number to find where they drop.
Topic 2 of 3 in Foundations of Context Engineering
Test yourself on Foundations of Context Engineering
The full Databricks Context Engineer Associate question bank has 600 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Databricks. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.