Part 3 of 4 · 2 min read
Exercises
Easy
Easy 1: Run it (30 minutes)
Run python -m pytest -q and python run_all.py in code/mini-ui.
Done when: you can explain the three view modes.
Easy 2: Your own events (30 minutes)
Feed a real trace from an agent you built into build_view.
Done when: you list what the grouped view hides and judge whether anything important is missing.
Hint
keep the critical-event invariant when you add your own event kinds.
Easy 3: Classify your actions (30 minutes)
List ten actions of an agent you know and tier each as auto, ask or confirm.
Done when: each has a reversible flag and a one-line reason.
Medium
Medium 1: Build the UI (4 hours)
Render the four layers in a small web page driven by the event stream.
Done when: a pending approval stays visible while the timeline is collapsed.
Hint
render from a pure function of events so you can unit-test the layers.
Medium 2: Smart resume (3 hours)
Add a "what happened while you were away" summary on resuming a paused run.
Done when: a test shows the summary lists changes and pending approvals.
Medium 3: Cost-aware start (3 hours)
Estimate a run's cost before starting and ask for confirmation above a limit.
Done when: a test covers both sides of the limit.
Hint
show a range, not a single number, because estimates are uncertain.
Hard
Hard 1: Usability test (8 hours)
Run the raw and layered views with five real users, planting one risky approval. (I did not run this, so your numbers are the ground truth.)
Done when: you report completion time, catches and trust ratings.
Hint
plant the risky approval in the middle of a run, not at the start.
Hard 2: Calibrate then phrase (8 hours)
Measure the calibration of a real model on your task and choose the wording thresholds from it. (I did not run this.)
Done when: a reliability table shows the words match the observed accuracy.
Hard 3: Approval fatigue study (8 hours)
Measure how prompt count affects catch rate with real participants. (I did not run this.)
Done when: you replace my assumed decay with a fitted one.