Part 3 of 4 · 2 min read
Exercises
Easy
Easy 1: Run it (30 minutes)
Run python -m pytest -q and python run_all.py in code/mini-tradeoff.
Done when: you can explain the 14.3 percent break-even.
Easy 2: Fill the sentence (1 hour)
Write the four-slot sentence ("I chose X over Y because...") for three decisions in your capstone.
Done when: each has a number and a reversal trigger.
Easy 3: Name the hard requirements (30 minutes)
For one decision, list the requirements an option must meet to be considered.
Done when: at least one option fails at least one requirement.
Medium
Medium 1: Your own matrix (3 hours)
Put your capstone's design options into matrix.py with a fixed scale, run the weight uncertainty and report the critical weights.
Done when: you can state how wrong your weights must be to flip the choice.
Medium 2: Write an ADR (2 hours)
Write a decision record for one real choice and run lint.
Done when: there are no flags and the revisit trigger is measurable.
Medium 3: Reproduce the rank reversal (2 hours)
Find a different three-option example where min-max flips the winner and fixed targets do not.
Done when: you have a test for it.
Hard
Hard 1: Measure your mix (8 hours)
Take 200 real or realistic requests, label each tool-heavy or simple, and compare a direct model with an agent on both groups with intervals. (I did not run this, so your numbers are the ground truth.)
Done when: you place your traffic on the break-even chart with an interval on the share.
Hard 2: Price of a point (6 hours)
Estimate what a one-point quality improvement is worth for your use case (fewer escalations, retained users) and compare it with the break-even cost. (I did not run this.)
Done when: you state the value, its assumptions and whether the agent is worth it.
Hard 3: Multi-stakeholder decision (6 hours)
Give three stakeholders different weights and find a design that is acceptable to all, or show that none exists. (I did not run this.)
Done when: you report each stakeholder's critical weights and the overlap.