Part 6 of 7 · 3 min read
Exercises
Each level has three exercises. Do the easy ones first, and try each one before opening its hint.
Easy
Easy 1: Read the schema (15 minutes)
Print Ticket.model_json_schema(). Find where the allowed priorities, the string length limits and the required fields appear.
Done when: you can point to each rule in the output.
Easy 2: Fixable in code, or ask again? (15 minutes)
Decide for each reply whether code can fix it or the model must try again: (a) a reply starting "Here you go:", (b) "priority": "urgent", (c) a trailing comma, (d) a missing closing brace at the end, (e) "needs_human": "no".
Done when: each has an answer and one reason, and you checked them with parse_reply.
Hint
parse_reply(text, Ticket).kind tells you which class the reply falls into.
Easy 3: Strict or forgiving (20 minutes)
For a bool, an int and a str field, try the values "1", 1, true, "true" and 1.0 under normal and strict types. Write down which are accepted.
Done when: you have a table of 15 results.
Medium
Medium 1: Your own schema (60 minutes)
Design a Pydantic class for a task of your choice, for example extracting a meeting from an email: title, start time, attendees, and a location or none. Write the schema prompt, six test replies (two valid and four with different mistakes) and tests that classify them.
Done when: six tests pass and each failure message names the field.
Medium 2: A nested object (60 minutes)
Extend the class with a list of line items, each with a name, a quantity above zero and a price. Check that the error message for a bad item includes its position, for example items.2.quantity.
Done when: a test with a bad third item produces a message naming items.2.quantity.
Hint
e.errors()[i]['loc'] is a tuple such as ('items', 2, 'quantity').
Medium 3: Failure statistics (60 minutes)
Run get_structured on 100 inputs with a pretend model that fails randomly but in a repeatable way (use a seeded random generator). Report the share of calls that needed 1, 2 and 3 attempts, the average number of calls, and the final valid rate for max_attempts of 1, 2 and 3.
Done when: you have a table, and you can say where extra retries stop paying off.
Hint
seed a random.Random so that your table is the same every run.
Hard
Hard 1: A tolerant extractor (90 minutes)
Decide what the extractor should repair and what it should refuse. Write ten messy replies (fences in other languages, two JSON objects, JSON inside a list, comments). For each, decide to accept, retry or reject, implement the decisions, and write down why each is safe.
Done when: all ten are tested and the rules are written down.
Hint
rejecting is a legitimate answer for any guess that could be dangerous.
Hard 2: Repair with care (120 minutes)
light_repair uses a pattern that could also change text inside a string value, for example "x, }". Write a failing test that shows it, then replace the repair with a small loop that knows when it is inside a string.
Done when: the failing test now passes and all earlier tests still pass.
Hint
track in_str and esc as extract_json does, and drop a comma only outside strings, when the next non-space character is } or ].
Hard 3: Streaming plus validation (120 minutes)
Combine this module with module 2. Stream a reply and show a preview, but only when the stream finishes run the full pipeline. If validation fails, retry quietly while showing "checking...", then show the final result. Test that the user never sees an invalid final object and that a dropped stream is retried.
Done when: tests cover a good stream, a stream that fails validation once, and a dropped stream.