Interview questions and limits: Files and scratchpads

Covers: Interview questions, with the follow-ups; Common mistakes; What the example does not do.

Interview questions, with the follow-ups

How do you stop tool output from filling the context window?

"I don't keep it in the window. I write the full output to a file in a sandboxed workspace and leave a stub with the path, the size and the first line. If the agent needs a detail it greps or reads a line range, with a cap on the result. In my test, twelve outputs of about a thousand tokens answered everything correctly for about 2,600 tokens, against about 74,000 for keeping them all, and the cost stayed flat when the outputs got four times longer." Follow-up: "Isn't an extra read a cost?" Yes: 99 tokens over six questions here. It is paid only when asked, and it is small compared with carrying every output on every step. Follow-up: "What if the model doesn't know to look?" Then the stub needs to say what the file contains; a title is the minimum, and a short plan or notes entry helps.

Scratchpad or file store: which, and when?

"A scratchpad is for what the agent judged important when it wrote it: decisions, findings, task status. It's always in the window, so it has to stay small, and it can't answer questions about anything unflagged. A file store holds the raw data and answers anything, at the cost of a read. In my test notes alone scored 50 percent, answering every flagged question and no unflagged one, and files scored 100. I'd offload by default and add notes only for facts that are needed repeatedly." Follow-up: "Who decides what to flag?" The model, at write time, with the errors that implies; that is why the raw data stays on disk.

What are the security risks of giving an agent file tools?

"Path traversal, symlinks, unbounded reads, partial writes, and hostile contents. I resolve every path and require it to be under the root, which also catches symlinks; cap every read; write to a temp file and rename; and treat anything read back as untrusted text that gets fenced in the window and cannot write memory. Beyond that I'd run the workspace in a sandbox with quotas and no network, because the path check protects the file tools, not the machine."

Common mistakes

A stub that contains the answer, or no title at all. The first wastes the saving; the second makes the agent open every file.
Letting read return the whole file. One call undoes the saving.
Filling notes with raw output. Notes are in every request; keep decisions and findings.
Trusting ".." not in path. Symlinks and a/../../x get through; resolve first.
Writing in place. A crash or a concurrent reader sees half a file.
Treating file contents as instructions. A file the agent saved from a web page is still a web page.
Offloading things the agent needs on every step. The stub then costs a read each time; keep those in the window.

What the example does not do

The agent is deterministic. A real model sometimes greps the wrong file, uses a bad pattern, or reads too much; the exercises measure that.
Questions are answerable by one grep. Multi-hop questions over several files, or ones needing a computation across files, cost more.
No concurrency, quotas or sandboxing.
Tokens are estimated by the module 1 counter.
Optional real-model experiment (I did not run this): give a real model the stub window and the three file tools and measure how often it picks the right file and pattern.

Check yourself

An agent saves a 1,000-token log to a file and leaves a 25-token stub. Later it needs a reference code that nobody flagged. Which is safer, a notes file or the saved log, and why? (The saved log. Notes only hold what was judged important when written; the log still has the detail.)

For the curious Four small fixes found in the example code

Reading a missing file, a directory or a binary file raised raw Python errors instead of an error the agent can be told about. A regular expression with a mistake crashed the search. A single line longer than the read cap returned nothing at all, only the word truncated. A failed write left a temporary file behind. All four are fixed and tested (63 tests).

What you will use daily at work

Practice	What it looks like
Stub plus path	The window holds a pointer, the file holds the data.
Capped reads	A read returns a few hundred tokens and says how to continue.
Confine the path	Resolve the path and check it is under the root before every read or write.

Draft: confirm where agent files may be stored on your platform and who can read them.

What you must be able to say (one page)

A workspace is a directory the agent reads and writes through a small set of confined tools. A scratchpad is a file the agent uses as working notes.
Offloading replaces a large tool output with a stub (path, size, first line, about 25 tokens) and saves the full text to a file. The agent fetches only the lines it needs with grep or a ranged read. Nothing is lost; the cost moves from every step to the steps that ask.
A scratchpad file (notes) keeps what the agent judged important at write time. It is cheap and always in the window, but it cannot answer a question about a detail nobody flagged. The file store can.
Quote this result: over 12 tool outputs of about 1,000 tokens each, keeping everything inline answered 100 percent in about 73,800 tokens; a 3,000-token inline window answered 10 percent; notes only answered 50 percent in about 1,050 tokens; offloading to files answered 100 percent in about 2,560 tokens (29 times fewer than keep-all); offloading plus notes answered 100 percent in about 3,560.
Offload cost depends on what is asked, not on how big the data is: at 400-line outputs, keep-all was refused and cost 242,000 tokens, while offloading stayed near 2,600.
A file tool is a security boundary. Confine every path to the root (resolve .. and symlinks), cap every read, write atomically, and treat file contents as untrusted input when they return to the window.