Part 6 of 7 · 3 min read
Exercises
Try each exercise before opening its hint.
Easy
Easy 1: Run and read (20 minutes)
In code/testing-demo, run pytest -vv. For each of the 13 tests write one line saying what behaviour it protects. Then break truncate_words (use words[1:max_words]) and see which tests fail first.
Done when: you can explain why the parametrised cases pinpoint the bug.
Easy 2: Name the missing test (15 minutes)
Look at the function truncate_words and the list of its tests. Write down three inputs nobody tested yet and what you expect each to return.
Done when: you have three inputs and three expected outputs, each different in kind.
Hint
think about empty input, a single word, and text with repeated spaces.
Easy 3: Good test or bad test? (15 minutes)
Say what is wrong with each: (a) a test that calls the real model API and asserts the reply is exactly "Paris", (b) a test with no assert, (c) a test that passes only if run after another test, (d) a test that sleeps for five seconds.
Done when: you named a fix for each of the four.
Medium
Medium 1: Add edge cases (25 minutes)
Add three more rows to the truncate_words parametrised test: text with tabs and newlines, a limit of 0, and a limit larger than the text. Fix the function if one fails.
Done when: all rows pass and you can explain what you decided for limit=0.
Hint
decide whether limit=0 returns an empty string or raises an error, then write the test for that decision.
Medium 2: A new endpoint with failure tests first (45 minutes)
Add POST /count that returns {"words": n} for the posted text, with a maximum of 20,000 characters. Write the failure tests before the code: empty text, too-long text, wrong type ({"text": 5}), missing field. Then make them pass.
Done when: four failure tests and one happy-path test pass.
Hint
Pydantic's Field(max_length=...) gives you the "too long" check for free; the test still matters.
Medium 3: Fake time (40 minutes)
Use the get_json function from module 3. Write a test proving that on a 429 reply with Retry-After: 3 it sleeps exactly 3 seconds, without actually waiting, by injecting a fake sleep that records its arguments.
Done when: the test runs in milliseconds and fails if you break the Retry-After handling.
Hint
get_json(..., sleep=waits.append) collects the delays in a list.
Hard
Hard 1: A scripted fake model (60 minutes)
Write ScriptedModel(replies: list[str]) with a complete(prompt) -> str method that returns replies in order and raises AssertionError("script exhausted") when called too many times. Then write answer_with_retry(model, prompt) that calls the model, and if the reply is not valid JSON, calls it once more with "Reply with valid JSON only." Test three cases: valid first time, invalid then valid, invalid twice (should raise).
Done when: the three tests pass and the test for "invalid twice" checks that the model was called exactly twice.
Hint
count calls by storing them on the fake (self.prompts.append(prompt)).
Hard 2: Mutation by hand (45 minutes)
Make five small deliberate bugs in summarizer.py one at a time (off-by-one, wrong comparison, dropped strip, wrong default, swapped argument). For each, run the suite and record whether any test fails. For every bug that survives, write the test that kills it.
Done when: every surviving bug has a new test that fails on the bug and passes on the original.
Hint
git restore summarizer.py puts the original back between mutations.
Hard 3: Test an agent loop without a model (60 minutes)
Write a tiny loop run(model, tools, task, max_steps) where the model returns either {"tool": name, "args": {...}} or {"final": text}. Using ScriptedModel, test: a two-tool-call run that ends with a final answer, an unknown tool name (the loop must report it back and continue), and a model that never finishes (the loop must stop at max_steps).
Done when: three tests pass without any network access, and the runaway test proves the step cap.
Hint
have the loop return a small result object (status, steps, messages) so tests can assert on more than the final text.