Part 5 of 7 · 4 min read
What the example does not do
Covers: What the example does not do; A checklist for "managing context in a long-running agent".
What the example does not do
- The summariser is an imitation. Real summarisers do better on some lines and worse on others; they paraphrase rather than copy; and their keep rates depend on the prompt. Only the structural result (decay with rounds, flat verbatim carry-over) is claimed.
- The user's original question is not modelled, though a real agent would keep it in every strategy. A real preserved set would raise the
truncateandclear_toolscores somewhat; it would not change the ordering for facts stated after the first message. - Scoring is by presence, not by whether a model uses the value correctly. Position effects ("lost in the middle", module 6) are not modelled.
- The extractor is patterns. Real structured state needs a model to extract, which brings back a miss rate and a cost; the exercises ask you to measure it.
- Caching is not simulated, and neither is the price difference between input and output tokens.
- Tools are assumed re-runnable. Clearing a result is only reversible if the tool is idempotent and the data still exists; a "read the current stock price" result is not safely re-runnable.
- Five seeds, one task shape. The numbers are exact for this setup and say nothing about the variance of a real agent.
A checklist for "managing context in a long-running agent"
- Keep an immutable event log; compute the request as a projection.
- Set a budget below the hard limit and a trigger at the budget; do nothing below it.
- Compact first: stub processed tool outputs and write-tool arguments, saying how much and how to restore.
- Keep a preserved set: system prompt, the user's original request, the last K turns.
- Carry decisions, constraints and identifiers as keyed, verbatim state with supersession.
- Summarise last, with a cap; know that a summary that feeds a summary decays; avoid regenerating identifiers.
- Record provenance and let only trusted roles write state; quarantine the rest; fence untrusted raw text.
- Count the whole cost: window per step plus summariser input and output; account for cache misses.
- Measure with probes for each fact class; track missing, stale and wrong separately.
Say it in one minute
"I keep an immutable event log and build the request as a projection of it. Below a token threshold I do nothing, which keeps the prompt cache warm. Above it, I compact first: processed tool results and write-tool arguments become stubs that say how much was removed and how to restore it. That's deterministic, free and reversible. If I'm still over, I carry decisions, constraints and identifiers as a keyed state block, copied verbatim, with newer values superseding older ones, and only trusted roles can write it. I summarise with a model last, because it costs a call, it's lossy, and a rolling summary decays geometrically with the number of rounds. I'd evaluate with probes for each fact class and report missing, stale and wrong separately. In my simulation at a 2,000-token budget that was 96 percent for structured state against 42 percent for a rolling summary, and the rolling summary also carried an injected 'skip the security review' line into trusted text in 7 of 20 probes."
Check yourself
After 60 rounds of rolling summary, the agent no longer knows a fact from step 3. Why, and what changes it? (Each round re-summarises the last summary, so early facts lose detail geometrically. Keep important facts in a structured state that is carried verbatim, not re-summarised.)
For the curious Summary laundering
A summariser that does not track where text came from can turn an untrusted line from a tool result into a plain statement in the summary, which the model then trusts. A structured state that records the source of each item, and only accepts items from trusted sources, does not have this problem. The cost is that you must decide the fields in advance.
What you will use daily at work
| Practice | What it looks like |
|---|---|
| Clear old tool output first | The largest, lowest-value part of the history, and it can be fetched again. |
| Trigger by size | Reduce when the request passes a threshold, not on every step. |
| Count the summariser | Its own input and output tokens are part of the cost. |
Draft: confirm whether your provider offers built-in context editing or compaction, and its current limits.