Can you stop an AI agent being tricked?

OpenAI says prompt injection is 'unlikely to ever be fully solved'. In one study, adaptive attacks got past 12 published defences, most at over 90 percent. OWASP and Meta advise limiting what the agent can do; no source measures it. Read 2026-11-06.

By Mackenzie Bowes · mackenzie@mackenziebowes.com · @_mackenziebowes on X

Contents

Short answer, from the sources below: nobody who has published on this claims a filter that reliably stops an AI agent from being tricked by text it reads, and two papers report filters being got round at high rates. The advice that OWASP, OpenAI and Meta give in common is about what the agent is allowed to do: fewer tools, narrower permissions, a person approving the risky step. No source here tests these controls on a small business's own agent.

What the risk is

An agent reads text (an email, a web page, a document) and acts on it. If that text contains instructions, the model may follow them as if the owner had written them. OWASP, the security-practice nonprofit, calls this prompt injection and says it can "enable unauthorized access" or "influence critical decisions". On whether it can be prevented, its page says: "it is unclear if there are fool-proof methods of prevention for prompt injection."[1]

OWASP separates a second problem, which is what makes an injection costly: "Excessive Agency", where the agent has more power than the job needs. Its page names three root causes: "excessive functionality", "excessive permissions" and "excessive autonomy".[2] Its example of excessive permissions is an extension meant to read data that "connects to a database server using an identity that not only has SELECT permissions, but also UPDATE, INSERT and DELETE permissions."[2]

EchoLeak (CVE-2025-32711) was, in its authors' words, "a zero-click prompt injection vulnerability in Microsoft 365 Copilot that enabled remote, unauthenticated data exfiltration via a single crafted email."[5] The US National Vulnerability Database record describes it as "Ai command injection in M365 Copilot allows an unauthorized attacker to disclose information over a network", and lists a score of 9.3 CRITICAL from Microsoft and 7.5 HIGH from NIST.[6] The paper reports that the attack went through by "evading Microsofts XPIA (Cross Prompt Injection Attempt) classifier", which is a guardrail of the kind described below.[5]

What a guardrail is and what the products offer

A guardrail is a check that runs on what goes into the agent or comes out of it. OpenAI's Agents SDK documents three placements: input guardrails, output guardrails and tool guardrails. Its page says "Input guardrails run only for the first agent in the chain" and "Output guardrails run only for the agent that produces the final output", so a chain of several agents is only partly covered unless tool guardrails are added.[9] The default mode runs the check alongside the agent to save time, and the page warns that "if the guardrail's tripwire is triggered, the agent may have already consumed tokens and executed tools before being cancelled."[9]

OpenAI's guardrails library page lists the checks on offer, with an estimated delay for each. Its jailbreak check "Detects attempts to jailbreak LLM calls via role-playing, system prompt overrides and injections", at an estimated 1,000 ms. A check for injection in tool calls, labelled Experimental, "Detects when function calls or function call outputs are not aligned with the user prompt's intent using LLM-based analysis."[10] Most of its checks are run by a language model, which can be tricked as the agent can. The page gives no detection rates.

Four other guardrail tools and what each says it does

How well do guardrails hold up?

Two studies attacked them directly.

A January 2026 survey of 78 studies on coding assistants reports that attack success rates against state-of-the-art defences "exceed 85% when adaptive attack strategies are employed."[11] That is a summary of other people's results, so it adds no new measurement.

The vendors say similar things in their own words. OpenAI's post on its Atlas browser agent says prompt injection "is unlikely to ever be fully" solved, comparing it to scams aimed at people, and describes an attack its own automated attacker found: a malicious email led the agent, which was asked to draft an out-of-office reply, to send a resignation letter, because it "treats the injected prompt as authoritative, and follows it."[7] Meta's page calls prompt injection a "fundamental, unsolved weakness in all LLMs".[8]

What the sources suggest instead

Meta's "Agents Rule of Two" is a design rule, not a filter. It says that "until robustness research allows us to reliably detect and refuse prompt injection, agents must satisfy no more than two of the following three properties within a session": [A] the agent can process untrustworthy inputs, [B] it can have access to sensitive systems or private data, [C] it can change state or communicate externally.[8] If a job needs all three, "the agent should not be permitted to operate autonomously" and needs supervision, such as human approval.[8] A mailbox assistant that reads incoming email (A), sees the whole inbox (B) and can send mail (C) hits all three.

OWASP's lists point the same way. Against excessive agency it advises minimising the extensions an agent can call, avoiding open-ended ones, minimising their permissions, running actions in the user's own context, and requiring approval. It adds that downstream systems should "implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed."[2] Against injection it lists constraining the model's behaviour, least-privilege access, and "Implement human-in-the-loop controls for privileged operations to prevent unauthorized actions."[1]

These are recommendations from standards bodies and vendors. None of the sources measures how many injection attacks they stop, or what they cost in staff time when a person must approve each step.

You can't narrow what you haven't written down. The AI setup sprint sets up one job on your own accounts and hands you a sheet of every piece of client data it touches, who can see it and which AI model reads it.

Not yet proven

If you run an agent that reads outside email or web pages and has measured what happens when someone tries to trick it, I would like to compare notes: mackenzie@mackenziebowes.com.

Sources

  1. OWASP Gen AI Security Project, LLM01:2025 Prompt Injection. Read in Rotunda 2026-11-06. genai.owasp.org/llmrisk/llm01-prompt-injection/
  2. OWASP Gen AI Security Project, LLM06:2025 Excessive Agency. Read in Rotunda 2026-11-06. genai.owasp.org/llmrisk/llm062025-excessive-agency/
  3. Nasr et al., The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections, arXiv 2510.09023, 10 October 2025 (abstract). Read in Rotunda 2026-11-06. arxiv.org/abs/2510.09023
  4. Hackett et al., Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems, arXiv 2504.11168 v3, 14 July 2025 (abstract). Read in Rotunda 2026-11-06. arxiv.org/abs/2504.11168
  5. Reddy and Gujral, EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System, arXiv 2509.10540, 6 September 2025 (abstract). Read in Rotunda 2026-11-06. arxiv.org/abs/2509.10540
  6. NIST National Vulnerability Database, CVE-2025-32711. Read in Rotunda 2026-11-06. nvd.nist.gov/vuln/detail/cve-2025-32711
  7. OpenAI, Continuously hardening ChatGPT Atlas against prompt injection attacks. Read in Rotunda 2026-11-06. openai.com/index/hardening-atlas-against-prompt-injection/
  8. Meta, Agents Rule of Two: A Practical Approach to AI Agent Security, 31 October 2025. Read in Rotunda 2026-11-06. ai.meta.com/blog/practical-ai-agent-security/
  9. OpenAI Agents SDK documentation, Guardrails (v0.24.0). Read in Rotunda 2026-11-06. openai.github.io/openai-agents-python/guardrails/
  10. OpenAI Guardrails, list of input, output and agentic guardrails. Read in Rotunda 2026-11-06. guardrails.openai.com/
  11. Maloyan and Namiot, Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems, arXiv 2601.17548, 24 January 2026, a survey of 78 studies. Read in Rotunda 2026-11-06. arxiv.org/html/2601.17548v1
  12. NVIDIA NeMo Guardrails library, GitHub readme (release 0.24.1). Read in Rotunda 2026-11-06. github.com/NVIDIA-NeMo/Guardrails
  13. Guardrails AI, GitHub readme. Read in Rotunda 2026-11-06. github.com/guardrails-ai/guardrails
  14. NVIDIA Build, model card for Meta's Llama Guard 4 12B. Read in Rotunda 2026-11-06. build.nvidia.com/meta/llama-guard-4-12b/modelcard
  15. Anthropic, Mitigating the risk of prompt injections in browser use. A vendor's own results. Read in Rotunda 2026-11-06. www.anthropic.com/research/prompt-injection-defenses

Terms on this page

  • AI agent: A system where a language model decides for itself how to do a task and which tools to use.
  • Human in the loop: A person takes part in an automated system's work, supervision or decisions.
  • Prompt injection: Text that makes an AI model do something its owner did not intend.
  • Tool call: A request from an AI model to run one action, such as a search or a file write, which the surrounding program carries out.