Part 1 of 4 · 1 min read
Overview
A chatbot that is tricked says something embarrassing. An agent that is tricked uses its tools. Interviewers ask "how do you secure an agent that reads the web or email?" The expected answer: assume the injection will sometimes work, and design so it cannot do damage.
About the example, and what is real
Real: the attack corpus (24 hand-written strings in three tiers, aimed at an example.test address), the 10 benign controls, the two filters, the session and tool-flag model, the rule-of-two policy, the quarantine extractor and the 13 tests in code/mini-injection. Assumed: the imitation agent obeys any instruction-like text that names an address with a probability I set (0.3 or 1.0); legitimate tasks are 20 percent "needs all three". Not real: no language model was attacked; the toy agent cannot decode base64 or ROT13, so some obfuscated attacks are harmless to it even though they bypass the filter. A real model may decode them. The attack strings are generic and non-operational.