AI browsing agents read this hidden text and may follow its instructions while appearing to respond normally. If developers use specific delimiters, attackers can spoof them. Attackers try to mimic system prompt formatting, hoping the model treats their input as privileged instructions. Anthropic dropped its direct prompt injection metric entirely in its February 2026 system card, arguing that indirect injection is the more relevant enterprise threat (Anthropic, 2026). 🧠 If the model lacks role-isolation enforcement, it may obey the user’s override. The core vulnerability that gives rise to prompt injection attacks lies in what can be termed the “semantic gap”.
Four properties make indirect injection a more pressing enterprise risk than direct injection. Repello’s research on RAG poisoning against Llama 3 demonstrated that targeted document injection caused consistent policy-violating outputs without any access to model weights, fine-tuning pipelines, or inference code. The model processes the complete token sequence including the hidden content; the guardrail inspects only the visible string. As documented in Repello’s research on emoji-based injection, variation selectors attached to Unicode characters can carry hidden instructions invisible to text-layer inspection. System prompts frequently contain business logic, API keys, confidential configuration, and internal data that organizations assume are hidden.
For developers, it’s important to treat setup instructions and scripts in repositories they’re unfamiliar with as untrusted code, ignoring what their agent may recommend. Our statistics are verified using a documented Research Process. Prompt injection has moved from a theoretical risk to a measurable, high-impact security threat shaping how organizations deploy and manage AI systems. A frequent global speaker, he engages https://uofa.ru/en/razrabotka-programmy-identifikacii-lichnosti-sovremennye/ at major technology and policy forums. His books are published by Springer, Cambridge, Wiley, Packt, and China Machine Press, including Generative AI Security, Agentic AI Theories and Practices, Beyond AI, and Securing AI Agents.
Data Exfiltration¶
In November 2025, Google’s DeepMind security team published research showing the scale of the problem. The vulnerability itself had been reported four months earlier by Jonathan Cefalu of security firm Preamble, who quietly disclosed it to OpenAI under the name “command injection.” He named it by analogy to SQL injection, the decades-old attack that broke websites by mixing user input with database commands. When a developer writes a system prompt like “You are a helpful customer https://leeds-welcome.com/the-future-is-now-top-trends-in-website-development-and-design-for-2023.html service bot for Chevrolet, only discuss our cars,” and a user types something, the model reads both as the same kind of input. AI Security, artificial intelligence, cybersecurity, data exfiltration, Generative AI, insider threat, machine learning, Prompt Injection, Vulnerability, zero-day “Models that would normally refuse harmful requests sometimes chose to blackmail, assist with corporate espionage, and even take some more extreme actions, when these behaviors were necessary to pursue their goals,” Anthropic said, calling the phenomenon agentic misalignment.
Logic Manipulation
Attackers can embed hidden commands within data sources, exploiting this ambiguity. Technical guardrails mitigate prompt injection attacks by distinguishing between task instructions and retrieved data. Additional safeguards include monitoring for hidden text in documents and restricting file types that may contain executable code, such as Python pickle files. Organizations can further mitigate risks by enforcing role-based data access and blocking untrusted sources. Additional techniques outlined by OWASP include enforcing least privilege access, requiring human oversight for sensitive operations, isolating external content, and conducting adversarial testing to identify vulnerabilities with tools like garak.
- Microsoft’s patches reportedly introduced options to restrict Copilot from using external communications in certain contexts.
- Standard guardrails that only inspect the user turn cannot stop indirect prompt injection, because the attack payload arrives through retrieved content rather than through user input.
- The majority of those defences originally reported near-zero attack success rates.
- At this point, in the scenario where the attacker only did Steps 1 and 2, data exfiltration was possible but not yet zero-click as the user would have to click that link for the attacker to receive the data.
Case 1: Slack AI Data Exfiltration
It requires no access to the target system, scales to affect every user who retrieves the poisoned content, leaves no trace in conversation logs, and is not covered by guardrails that only inspect the user turn. It is the highest-priority prompt injection risk for enterprise AI deployments because it requires https://www.daegu2011.org/category/technology/ no access to the target system and scales to affect every user whose session retrieves the poisoned content. Its context integrity layer flags outputs that diverge from stated user intent, which is the primary behavioral signal for a successful indirect injection that bypassed the input inspection layer. A deployment can be highly resistant to jailbreaking and fully exposed to indirect injection simultaneously.
Key Cybersecurity Insights & Evolving GenAI Risk Horizons
- The July 2025 Invisible Injections study found overall attack success rates of 24.3% across GPT-4V, Claude, and LLaVA, with neural steganographic methods reaching 31.8% .
- Zscaler’s ThreatLabz documented two real-world campaigns which used a technique called indirect prompt injection, where instructions are planted in content an AI agent reads, such as a web page, to steer its behavior.
- The attack highlights the need for layered, defense-in-depth security architectures that treat both inputs and outputs as untrusted, coupled with provenanceaware context isolation.
- Sophisticated injection attacks can bypass prompt filters through indirect delivery, creative phrasing, or multi-step manipulation.
Standard guardrails that only inspect the user turn cannot stop indirect prompt injection, because the attack payload arrives through retrieved content rather than through user input. For enterprise deployments, indirect prompt injection is generally the higher-priority risk. The NIST AI Risk Management Framework (AI RMF 1.0) addresses both classes under its Govern and Measure functions, specifically calling for adversarial testing that covers both user-turn and retrieved-content attack surfaces.
No responses yet