Part 3 of 4 · 2 min read
Exercises
Easy
Easy 1: Run it (30 minutes)
Run python -m pytest -q and python run_all.py in code/mini-dataset.
Done when: you can explain each table in the results.
Easy 2: Define a duplicate (30 minutes)
Write the duplicate definition for a dataset of your own (level, threshold, normalisation).
Done when: you list two borderline examples and your decision on each.
Easy 3: S-curve by hand (30 minutes)
Compute 1 - (1 - s^r)^b for s = 0.5, 0.7, 0.9 with 8 x 8 and 16 x 4.
Done when: your numbers match candidate_probability.
Hint
at s = 0.5, 8 x 8 gives about 0.03 and 16 x 4 about 0.64.
Medium
Medium 1: Tune bands and rows (3 hours)
Find the setting with the fewest candidates that still gives 99 percent recall at threshold 0.5.
Done when: you report candidates and recall for three settings.
Hint
more rows per band lowers false candidates; more bands raises recall.
Medium 2: Harder near-duplicates (3 hours)
Add edits that drop Jaccard below 0.5 and measure how recall falls.
Done when: you plot recall against true similarity.
Medium 3: Contamination with a perplexity check (3 hours)
Use a bigram language model's perplexity to flag leaked test rows and compare with the n-gram check.
Done when: you report precision and recall for both.
Hint
train the language model only on the training set.
Hard
Hard 1: Your own traces (8 hours)
Deduplicate and group-split a real set of agent traces and measure the effect on a fine-tune. (I did not run this, so your numbers are the ground truth.)
Done when: you report score gaps between random and group splits.
Hard 2: Embedding dedup (8 hours)
Add embedding-based near-duplicate detection and compare it with MinHash on paraphrased pairs. (I did not run this.)
Done when: you report recall, false matches and cost.
Hard 3: Synthetic data loop (8 hours)
Generate data with a small model, filter by tests or a judge, train, and repeat for three rounds with and without real data. (I did not run this.)
Done when: you track a rare-case metric across rounds.
Hint
choose a rare-case metric before round one.