Part 5 of 6 · 3 min read
Exercises
Easy
Easy 1: Run it and read the table (30 minutes)
In code/mini-workspace, run pytest -q, python demo.py and python run_all.py.
Done when: you report the total tokens and accuracy of offload and inline_all, and say in one sentence why notes_only scores 50 percent.
Hint
agent.table() returns the rows used in the notes.
Easy 2: Break the box (30 minutes)
Try to read ../secret, a/../../secret and a symlink you create inside the workspace that points to a file outside it.
Done when: all three raise WorkspaceError, and you say which of them a check for the substring ".." would have missed.
Hint
os.symlink(target, workspace.root / "link") creates the link.
Easy 3: Change the cap (30 minutes)
Change MAX_READ_TOKENS to 100 and to 2,000 and re-run.
Done when: you report how retrieval tokens and accuracy change and say what a cap that is too small does to an agent that needs a long passage.
Hint
workspace.MAX_READ_TOKENS is a module constant.
Medium
Medium 1: A plan file (2 hours)
Add plan.md with one line per task and a status. The agent must tick a task when finished and re-read the plan before each step. Add a task where repeating a step is the failure.
Done when: the agent without the plan repeats a step and the agent with it does not, and you report the token cost of the plan per step.
Medium 2: Smarter stubs (2 hours)
Include the first three lines and the count of lines matching RESULT in each stub. Compare stub size and accuracy with the first-line-only stub when two outputs share a title.
Done when: you show a case where the title-only stub picks the wrong file and the richer one does not, and report the extra tokens.
Hint
change agent.stub, not the workspace.
Medium 3: Two writers (3 hours)
Run two agents that append to the same notes.md. Show the lost-update problem with the current append (read, then write) and fix it.
Done when: a test that fails before the fix and passes after it, and you say what lock or append mode you used and why.
Hint
open the file with mode "a" so the operating system appends; or take a lock file.
Hard
Hard 1: A real model picks the file (4 hours)
Replace the deterministic file choice and pattern with a real model that sees only the stubs and the question. (I did not run this, so your numbers are the ground truth.)
Done when: you report the rate at which it picks the right file, the rate of a useful pattern, the extra tokens, the model's name and the date.
Hint
give the model grep(path, pattern) and read(path, start, n) as tools.
Hard 2: Hostile file contents (4 hours)
Plant a line in one output that instructs the agent to write a false value to notes. Measure how often the real agent follows it with and without fencing. (I did not run this.)
Done when: you report the rate in each condition and say which defence worked.
Hint
fence the returned text with the markers from module 1.
Hard 3: Multi-file questions (5 hours)
Add questions such as "which output has the highest total?" that need all twelve files. (I did not run this.)
Done when: you compare reading every file, a notes index that records every total, and a small script run in the workspace, and say which the agent should use.
Hint
a notes index is cheap if the agent writes one line per output as it arrives.