Context Compression and Compaction · Topic 2 of 2
Tuning Compaction, Trimming and the Cost Trade-Off
Context Engineer exam: tune a compaction prompt recall first, decide if trimming heuristics are enough, and weigh aggressive versus conservative modes.
- 5 min
- Tune compaction recall first, then precision, and measure each version on real traces.
- Stop removing content when task success drops; that is the precision limit.
- Trimming is rule-based with no model call; it suits tasks where relevance follows recency.
What it is
Once an agent's history needs to shrink, three choices remain. You pick a method (rule-based trimming or model-based compaction), you tune the compaction prompt if you use one, and you set how hard it compresses.
Trimming drops old messages or clears consumed tool results by fixed rules, with no model call. Compaction asks a model to summarize the history, then continues with the summary plus recent items. Each choice trades token cost against the risk of losing something a later step needs.
How it works
Tuning a compaction prompt: recall first, then precision
Start with a prompt that captures everything that could matter, even if the summary is long. A missing fact breaks the task, while an extra sentence only costs tokens, so recall is the safer first target.
Then iterate for precision: remove what the summary carries but no later step uses, such as copied tool payloads, repeated findings and narration.
| Iteration | Prompt change | Summary size | Resumed task success |
|---|---|---|---|
| 1, recall | Keep findings, decisions, constraints, rejected options, open tasks, IDs, tool results | 9,000 tokens | 94% |
| 2, precision | Replace tool results with their findings | 4,500 tokens | 94% |
| 3, precision | Drop step by step narration | 3,200 tokens | 93% |
| 4, too far | Drop rejected options | 2,600 tokens | 81% |
Iteration 3 is the stopping point here. Iteration 4 saves 600 more tokens but loses a category that later steps depended on, which shows up as a drop in task success.
To measure this, compact a set of saved traces with each prompt version, resume the task from the summary, and score the outcomes with mlflow.genai.evaluate() using scorers such as correctness. Compare versions on every metric and over repeated runs.
Is trimming enough? Read the relevance pattern
Hard-coded trimming heuristics, such as "keep the last N turns" or "clear tool results after they are used", are cheap, fast and predictable. They work when relevance follows recency, so older content really is no longer needed.
| Task relevance pattern | Trimming enough? | Why |
|---|---|---|
| Support chat where each question is independent | Yes | Old turns rarely matter |
| Tool heavy loop where each result is used once | Yes, clear consumed results | Findings live in later messages |
| Constraints set early, applied throughout | No | A recency rule drops the early constraint |
| Decisions in step 5 that step 40 must honor | No | Relevance is long lived and scattered |
| Mixed: a few standing rules plus bulky tool output | Combine | Trim tool output, compact or pin the rules |
When important content is scattered through the history and not concentrated at the end, trimming cannot tell it apart from filler. That task needs compaction, which reads the content and keeps it by meaning, or structured notes stored outside the window.
Aggressive versus conservative compaction
Aggressiveness is how much the summary shrinks the history. It sets both the ongoing token cost and the risk of losing subtle context.
| Aspect | Aggressive | Conservative |
|---|---|---|
| Summary size | Very small | Larger |
| Token cost per later call | Lower | Higher |
| Fidelity | Loses nuance, exact values, reasons | Keeps detail and wording |
| Good fit | Repetitive, low dependency tasks | Tasks whose later steps rely on subtle earlier details |
| Main risk | Coherence failures later | Window fills again sooner, higher spend |
A worked number: a 100,000 token history compacts to 2,000 tokens aggressively or 12,000 tokens conservatively. Over 30 more model calls that each resend the summary, aggressive saves about 300,000 input tokens.
That saving is worth it only if task quality holds. If a regulatory task needs an exact threshold from step 6, losing it once can cost more than the tokens saved. A common middle path compresses tool output aggressively and keeps decisions and constraints conservatively.
How the exam asks about this
- "A first compaction prompt keeps the agent coherent but the summary is large." Keep the categories and iterate for precision, removing superfluous outputs such as copied tool results.
- "A keep last N turns rule makes the agent forget constraints the user set at the start." Trimming does not fit this long lived relevance pattern; use compaction or persisted notes.
- "The team must cut cost, and the task depends on exact values from early steps." Prefer conservative compaction for those values, or keep them in structured form while compressing the rest.
What to remember for the exam
- Tune compaction recall first, then precision, and measure each version on real traces.
- Stop removing content when task success drops; that is the precision limit.
- Trimming is rule-based with no model call; it suits tasks where relevance follows recency.
- Long lived or scattered relevance needs compaction or notes stored outside the window.
- Aggressive compaction cuts per-call cost; conservative compaction keeps fidelity at higher cost.
- Compare prompt versions on all metrics and across repeated runs with MLflow evaluation.
Do not start tuning by making the summary as short as possible. A precision-first prompt silently drops categories you only discover when downstream steps fail, and the exam frames the order as recall first.
Common mistakes
Choosing trimming because it is cheaper. Trimming saves a model call, but if the task has standing constraints set early, it removes exactly what the agent needs.
Judging compaction by summary size alone. A smaller summary is only better if the resumed task still succeeds. Always pair the token count with a quality metric from evaluation.
Treating aggressiveness as all or nothing. The best setting often differs by content type. Compress bulky tool output hard and keep decisions, constraints and exact values in full.
Check yourself
Why tune a compaction prompt for recall before precision?
A missing fact can break later steps, while extra content only costs tokens, so you first make sure nothing needed is lost and then remove what no step uses.
When are hard-coded trimming heuristics sufficient?
When relevance follows recency, for example independent support turns or tool results used once. They fail when early content such as user constraints must stay in force.
What is the main risk of aggressive compaction?
Losing subtle details, such as exact values or reasons, that later steps need, which causes coherence failures despite the lower token cost.
Topic 2 of 2 in Context Compression and Compaction
Test yourself on Context Compression and Compaction
The full Databricks Context Engineer Associate question bank has 600 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Databricks. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.