Part 1 of 6 · 1 min read
Overview
The first production incident of an agent is usually an irreversible action nobody approved; the second is a reviewer who approves everything without reading. An interview answer needs both halves: what to gate, and how to build the pause.
About the example, and what is real
Real: the code in code/mini-hitl (five gating policies, a reviewer attention model, an event-driven simulation of one worker with blocking and non-blocking waits, an approval-rate check) and its 14 tests. Assumed: 30 percent of actions irreversible; harm chances of 4 percent (irreversible) and 0.5 percent (reversible); a reviewer who catches 90 percent of harm at full attention, with attention equal to 1 divided by (1 plus requests per hour over 20); a 2 percent false-reject rate; exponential waits; the agent's risk scores as described above. The fatigue formula is my invention to make a known effect visible; no real reviewer was measured, and I have no measured fatigue figure to compare it with. No model is called.