Part 3 of 4 · 2 min read
Exercises
Easy
Easy 1: Run it (30 minutes)
Run python -m pytest -q and python run_all.py in code/mini-guardrails.
Done when: you report the bad-input acceptance of both validators.
Easy 2: Write a schema (30 minutes)
Write a schema for a send_message(to, body) tool.
Done when: it rejects an empty recipient, an over-long body and an unknown field.
Easy 3: Find the bypass (30 minutes)
Replace fullmatch with match and $.
Done when: you show which test input now passes and why.
Hint
re.match(r"^ORD-\d{6}$", "ORD-123456\n") succeeds because $ matches before a final newline.
Medium
Medium 1: Concurrency (3 hours)
Run two refunds in parallel against one order.
Done when: you show the double refund, then fix it with a lock or transaction.
Hint
check and update the refunded total inside one transaction.
Medium 2: Measure correlation (4 hours)
Build three different filters and run one attack set.
Done when: you compute the overlap of their misses and the combined miss rate.
Hint
compute the set of attack ids each filter misses and intersect them.
Medium 3: Output guard evasions (3 hours)
Add five new evasions and update the guard.
Done when: you report false positives on a benign set of at least 50.
Hard
Hard 1: Replay logs (8 hours)
Replay a real agent's tool calls through the validator. (I did not run this, so your numbers are the ground truth.)
Done when: you report rejection counts by reason and fix one real bug they reveal.
Hard 2: Guardrail library (8 hours)
Compare a real guardrail library with your validator on the same cases. (I did not run this.)
Done when: you report accepted and rejected sets and latency.
Hard 3: Repair loop (6 hours)
Add a re-ask loop with a maximum of two retries. (I did not run this.)
Done when: you show that repaired output is re-validated and that the retry count is bounded.
Hint
count attempts per call and fail closed after the cap.