Passwords and look-alikes

Passwords and look-alikes

Without the vault the password sits in the agent's context, because the agent has to type it into a sign-in form, and a page that can send the agent to a look-alike (bank-example.test) gets it: the leak judge sees the password in a request to a host that does not own it. With the vault layer the agent types a token; the browser swaps the token for the password only in a password field on the host that owns it (the planted defect vault_any_host fills it anywhere, and is caught by two checks). The agent never holds the password, so no sentence in a page can make it say it. Anthropic's guidance (the documents I read) is the same in spirit: keep credentials and sensitive data away from the model, and give the agent a minimal, separate environment, with human confirmation for consequential actions such as payments and accepting terms. The computer-use page also says to restrict internet access to an allowlist of domains, which is the host layer here.

A leak detector can also be wrong in the other direction: sending your own password to your own bank is not a leak, and the check (C27) fails a defect that flags it.

The detector is the weakest layer, and why

The detector is a pattern list: it removes lines containing "assistant:" or "AI agent:" and phrases such as "ignore all". It is a stand-in for a trained classifier, and it shows why one pattern list is not one.

• It catches all 6 canonical wordings and none of the 6 paraphrases ("Note to AI", "Dear AI model", "Attention AI" instead of "Assistant:"). With the detector as the only layer, all 180 harming cases happen against the paraphrases.
• It flags 5 of 12 ordinary lines: "Customer assistant: Priya will help you at the desk", "Please ignore all the noise in the video", a book title containing "AI agents:", and two more. A defect that makes it broader (any "note", "ai" or "ignore") is caught by a check because it removes ordinary text.
• A defect that makes it case-sensitive is caught only because my sample attack starts with a capital letter. I predicted nothing would catch it. A lower-case "assistant:" would get through.

Real classifiers are models and do much better against rewording, and Anthropic reports using one on content before it reaches the model. They still miss things, and a classifier is a probability, not a wall. The hygiene, host and contract layers do not depend on understanding the attacker's sentence, and that is why they are the layers I would build first.

What the model-side defences add, and what I have not measured

Everything above is outside the model. Anthropic's research page describes three layers inside and around the model: training the model on injections (reinforcement learning), classifiers over untrusted content, and human red-teaming. It reports attack success rates against an adaptive attacker (a best-of-N search with 100 attempts per environment, about 2,200 to 3,200 attempts per figure). These are their numbers on their test, not mine:

Three honest readings. A model that resists is far from one that always obeys (my stand-in is the second): single-digit and low-double-digit percentages against a persistent attacker, falling to about one to two percent with extra safeguards. Even the best number is not zero: the page says plainly that no browser agent is immune and that the web is adversarial. And the numbers are for those models, those tests and that date, which is why a harness that holds even when the model fails is the right thing to build. One percent of a million tasks is ten thousand incidents.

Does the suite work? Planted defects

The harness has 30 checks (one more compares my page model with Chromium and needs a Chromium installed; all 31 pass on the correct harness). I switched on 31 realistic defects, one at a time: a view that lists hidden elements, clicks that skip the scale-back, a host check that accepts suffixes, a contract that ignores parameters, a confirmation that shows the agent's words, a vault that fills anywhere, and so on.

• 28 of 31 were caught; 28 matched my prediction exactly (25 caught as predicted, and the 3 escapes below, which I predicted).
• Three escaped (I thought of them after writing the checks, and wrote down that I expected no check to notice): hygiene_drops_aria_hidden (hygiene throws away text that is drawn on screen), confirm_total_wrong (the order prompt shows a total of zero) and step_cap_off_by_one (the cap allows one step too many). These are real holes in the suite. Closing them is exercise Easy 3.
• One prediction was wrong: detector_case_sensitive. I expected it to escape; check C22 caught it, for the accidental reason in section 10.
• Two had an extra failure: a form that forgets prefilled values (form_drops_prefill) and a mail prompt without the recipient (confirm_hides_recipient) both also failed the check that a legitimate reply asks the person once, because the prompt or the form is part of that legitimate path.

One more finding came from a check I wrote wrongly. My first test for "an agent that repeats a failing click stops" used an agent that ignored a cookie banner. It did not loop: it gave up straight away, because it could not read the prices under the banner. I replaced it with a plan that repeats one action thirty times. A check has to produce the situation it claims to test.

What the example does not do

• No real page structure. No frames, shadow DOM, pop-up windows, redirects, downloads, file dialogs, drag and drop, keyboard shortcuts, or time. Any of them can change the results.
• No CAPTCHAs, no two-factor sign-in. A real agent should stop and hand these to the person; bypassing bot detection is not something to build.
• A scripted agent. No model reasons, mis-clicks for its own reasons, or recovers from a surprise. The conclusions about a real model's behaviour are mine.
• A perfect reader of screenshots. Real models misread small text, which makes the screenshot results friendlier than they would be.
• One worst-case attacker. It obeys every sentence it can parse. It does not adapt to the layers, it does not try to split an instruction across pages, and it does not use images with embedded text beyond one canvas.
• A "person" that is a script. The fatigue result is a model of behaviour, not a measurement.
• Desktop applications. Computer use on a native app has no page source, and the accessibility interface differs by operating system. My example is a browser world. The computer-use documentation I read says to run it in a dedicated virtual machine or container with minimal privileges, and Claude Code's own documentation (read for module 8) says computer use runs on your actual desktop, not isolated, which is the opposite of the sandbox of module 8. Putting a browser agent inside the module 8 sandbox is a good idea and is not drawn here.