Part 5 of 6 · 3 min read
Exercises
Easy
Easy 1: Run it and read the tables (30 minutes)
In code/mini-explore, run pytest -q and python run_all.py.
Done when: you report best-of-N and majority accuracy at N=5 with their calls, and say why the majority vote flattens.
Hint
sampling.best_of_n(n, params, tasks, seed) and sampling.majority(...).
Easy 2: Compute the cost of N (30 minutes)
For a task that costs 3,000 tokens per attempt and 500 per verification, compute the tokens for best-of-N at N = 1, 3, 5, 9.
Done when: you tabulate tokens and accuracy (from the table) and name the N where the gain per extra 10,000 tokens falls below one point.
Hint
tokens = N times 3,000 plus N times 500 for the verifier.
Easy 3: Order a queue by hand (30 minutes)
Take six tasks with values and costs and a budget, and order them by value first, cheapest first and value per cost.
Done when: you report the value achieved by each and check them against prioritise.optimal.
Hint
greedy by ratio, then check with prioritise.optimal(tasks, budget).
Medium
Medium 1: Break the vote (2 hours)
Sweep concentrate from 0 to 1 and N from 1 to 31.
Done when: you plot majority accuracy by N for three values of concentrate and say when increasing N stops helping.
Hint
replace(Params(), concentrate=c).
Medium 2: Calibrate the estimates (2 hours)
Make value estimates noisy with a bias (the agent overrates cheap tasks) and compare the rules.
Done when: you show which rule is most robust to bias and which to random noise.
Hint
multiply the estimate by a factor that decreases with cost.
Medium 3: Discounting (3 hours)
Add Thompson sampling with a discount on old observations and test it with the best strategy changing at different times.
Done when: you report regret against plain Thompson sampling for early, middle and late changes, and say when discounting pays.
Hint
multiply wins and pulls by 0.98 before each update.
Hard
Hard 1: Real samples (5 hours)
Draw 15 samples each for 100 tasks with checkable answers and compute best-of-N and majority vote. (I did not run this, so your numbers are the ground truth.)
Done when: you report accuracy against N, the model and the date, the share of wrong answers that agree, and compare them with the simulation.
Hint
store the raw answers, so you can recompute agreement.
Hard 2: Learn which prompt works (4 hours)
Run a bandit over three real prompt variants for 200 tasks with an automatic success check. (I did not run this.)
Done when: you report the selection counts, regret against the best single variant, and the check's reliability.
Hint
use bandit.run's logic with real outcomes.
Hard 3: A small tree search (5 hours)
Implement beam search over candidate next steps for a multi-step task with a scorer. (I did not run this.)
Done when: you report success against beam width and calls, and compare it with best-of-N at the same number of calls.
Hint
keep the top w partial paths by score at each depth.