Part 3 of 4 · 2 min read
Exercises
Easy
Easy 1: Run it (30 minutes)
Run python -m pytest -q and python run_all.py in code/mini-sysdesign.
Done when: you can explain why the simulated p95 differs from the sum of p95s.
Easy 2: Write the targets (30 minutes)
For an agent you know, write four numeric targets (quality, p95, cost, availability).
Done when: each number has a reason.
Easy 3: Draw the baseline (30 minutes)
Draw the one-call baseline and list what each future box would be for.
Done when: every box names the target it serves.
Medium
Medium 1: Your own stages (3 hours)
Edit design.py with numbers from your project's traces and rerun.
Done when: you can name the first target that fails.
Medium 2: Sensitivity (3 hours)
Run sensitivity on your design and propose two changes ranked by saving.
Done when: each change names its quality or cost side-effect.
Medium 3: Mock design (3 hours)
Give yourself 35 minutes to answer "design a research agent" using the seven steps; record it.
Done when: you list the steps you skipped.
Hard
Hard 1: Correlated failures (8 hours)
Add a shared outage state (a provider down for 1% of the time) to the simulator and compare the availability with the independent model. (I did not run this, so your numbers are the ground truth.)
Done when: you report both availabilities and explain the gap.
Hard 2: Queueing (8 hours)
Model a fixed pool of workers with arrivals at a set rate and show how p95 grows with load. (I did not run this.)
Done when: you find the load at which the p95 target fails.
Hard 3: Quality in the loop (8 hours)
Evaluate a small and a large planner model on a labelled task set and use the measured quality in the design table. (I did not run this.)
Done when: the table's quality column is measured, with intervals.