Part 5 of 6 · 5 min read
Exercises
Easy
Easy 1: Run and read (20 minutes)
In code/mini-chunk, run python demo.py and pytest. For three rows of the table, write one sentence saying why the numbers differ from the row above (for example, why overlap raises intact at size 120 but not at size 500).
Done when: you have three sentences and each names a concrete cause (a cut answer, a mixed-topic chunk).
Easy 2: Count the chunks by hand (15 minutes)
For a text of 1,000 characters, size 200 and overlap 50, predict the number of chunks before running fixed. Then run it. Repeat for overlap 0.
Done when: your predictions match the code, and you can state the formula you used.
Hint
with step size - overlap, the number of chunks is ceil((length - overlap) / (size - overlap)), but check the last chunk.
Easy 3: Find the cut (20 minutes)
Write three sentences of your own where a fixed cut of 30 characters splits an important fact (a number from its unit, a name from its role). Chunk them with fixed and with sentences and show which one keeps the fact whole.
Done when: you have one example where fixed fails and sentences succeeds, and one where both fail because the fact spans two sentences.
Hint
numbers and their units ("90" and "seconds") are the usual casualties.
Medium
Medium 1: A markdown parser (45 minutes)
Add parse_markdown(text) -> list[Block] to parse.py: lines starting with #, ##, ### become headings with the right level; blank-line-separated text becomes paragraphs; lines starting with - or * become one block each; fenced code blocks (between three backticks) become a single block of kind code. Add at least four tests.
Done when: by_structure works on its output and a test shows a code block is never split across chunks.
Hint
handle the "inside a fence" state with a boolean while you loop over lines.
Medium 2: Count in words, not characters (40 minutes)
Write fixed_words(text, max_words, overlap_words) that chunks by words, and add a rough_tokens count (about one token per four characters, as in Level 1) to measure.evaluate so the report shows average tokens per chunk. Compare it with fixed at a matching size.
Done when: the report prints tokens, and a test shows no chunk from fixed_words ever cuts a word.
Hint
text.split() loses the original spacing; join with a single space and accept that.
Medium 3: Page numbers survive cleaning (50 minutes)
clean_pdf_pages returns one string and forgets which page each line came from. Change it (or add a sibling function) to return (line, page) pairs, and make a chunker that puts meta["page"] (the page where the chunk starts) on every chunk. Test with PDF_PAGES. Then add a case where the first line of a page is real content, not a header, and decide how your header rule should treat it.
Done when: chunks from page 2 report page 2, and you have a written decision about the header rule with a test.
Hint
the first line of a page is real content if it does not repeat on other pages; the example rule already uses a repeat count, so the case to test is a page whose first line is real content (the example now keeps a one-page document intact).
Hard
Hard 1: Run it on your own documents (90 minutes)
Take a real document set you know (your own README files, a public manual or a few help-center pages saved as HTML or text). Parse and clean it, read the output, and fix at least two problems you find. Write 15 questions with exact answer strings. Sweep three sizes and overlap 0 and 20 percent with at least two strategies. Report intact, top-1 and top-3 in a table and name the best setting with one sentence of reasoning. (I did not run this on your data, so your numbers are the ground truth.)
Done when: you have a table of at least 12 rows, two parsing fixes you can describe, and a conclusion that is allowed to say "the defaults were fine".
Hint
write the answers before looking at which chunks exist, and write at least three that span two sentences.
Hard 2: Repair two columns (75 minutes)
Many PDF extractors return words with positions. Invent a small list of (x, y, word) tuples for a two-column page with a title line spanning both columns. Write read_columns(words, page_width) that returns text in reading order: the full-width title first, then the whole left column top to bottom, then the whole right column. Test with at least three layouts, including one with a column that is empty.
Done when: the three tests pass and the naive "sort by y then x" order is shown to interleave the columns on your test page.
Hint
a line is "full width" if its words begin left of the middle and end right of it; split the remaining words at page_width / 2.
Hard 3: Small-to-big retrieval (90 minutes)
Index small chunks (about 120 characters) but, when a chunk is retrieved, return its parent section (the whole by_structure chunk). Add six new questions whose answers need a neighbouring sentence ("What happens to the guest password if I do nothing?"). Compare four systems on all 18 questions: small only, large only, structure only, and small-to-big, using a measure that checks whether the returned text contains the answer plus the context sentence.
Done when: you have a four-row table, a test that proves the parent link is correct, and one sentence on when small-to-big costs more than it gains (for example, larger prompts).
Hint
store parent_index in meta when building small chunks, then look the parent up after retrieval.