Prompt Injection Attacks in Production Systems

Prompt injection is a security flaw that can make a bot say something silly. It's a structural weakness in how large language models process text, and once an AI agent gets access to private data and the ability to reach the outside world, that weakness turns into a real enterprise threat. Understanding why it happens, and why no single fix closes the gap, is quickly becoming a baseline security skill, not a niche one.

Start with the mechanics. An LLM takes in one continuous stream of text, made up of the system prompt, the user's question, and whatever external content gets pulled in, whether that's a document, an email, or a webpage. All of it lands in the same context window as plain language tokens. The model has no internal wall separating "instructions I should trust" from "content I'm just supposed to read." It sees words, and it responds to whatever words carry the most persuasive instruction, regardless of where those words came from.

People reach for the SQL injection comparison because it's the closest thing security folks already know. Both exploits mix trusted commands and untrusted data in the same channel. But the comparison only goes so far, and here's where it breaks down: SQL injection got solved, structurally, with parameterized queries. You can build a query interface where user input literally cannot be read as a command. There is no equivalent fix at the model layer for prompt injection. Defenses exist, but they all sit at the application layer or the context layer, wrapped around the model. Nothing inside the model itself enforces the separation, which is worth sitting with before reading any further.

OWASP put prompt injection at the top of its list, LLM01:2025, the single highest-priority risk in its Top 10 for LLM applications. That ranking matters because OWASP is describing a persistent condition, not a bug some future patch will quietly retire. It's a structural feature of how these systems work. Early research laid out the two outcomes this flaw enables: goal hijacking, where an attacker forces the model to complete a task it was never meant to do, and prompt leaking, where the attacker gets the model to reveal hidden instructions or private data it was holding onto.

A national cybersecurity authority described LLMs, in a December 2025 assessment, as "inherently confusable deputies." That term is borrowed from older security research, and it means exactly what it sounds like: systems that can be tricked into acting against the people they're supposed to serve, because nothing inside them reliably tells trusted instruction apart from untrusted data. Every attack and every defense discussed from here on is a downstream consequence of that one property. Once that clicks, prompt injection stops looking like a content-moderation headache and starts looking like what it actually is.

The three attack types and where each one reaches the model

Three categories describe the three points where an attacker can actually touch the model.

Direct injection is the simplest: the attacker is the user, typing something like "ignore previous instructions and..." straight into the chat box. On its own, this is mostly a bounded nuisance, since the user is both the attacker and the only person affected. The story changes fast once that same chatbot gets wired up to tools. Give it the ability to send emails, run code, or move files, and a direct injection stops being an embarrassing output. It becomes the trigger for an action chain.

Indirect injection is where things get serious, worrying security teams most. Here, the malicious instructions aren't typed by the user at all. They're buried inside content the AI pulls in during a normal task, such as an email, a PDF, a webpage, a calendar invite, a code comment, or a row in a database. The user never sees the injected text. The model just reads it as part of doing its job, and follows whatever instructions are hiding inside.

This matters most for retrieval-augmented generation (RAG) systems, email-processing agents, coding assistants, and anything that treats outside content as routine input. Standard prompt filtering catches direct override attempts well; it's built for exactly that. It largely fails against indirect injection, because the malicious text is dressed up as ordinary data rather than an obvious command. Google researchers tracking the open web found a jump in malicious injection payloads embedded in web content between late 2025 and early 2026, and web-based indirect injection alone now accounts for a large share of documented LLM security incidents.

Stored injection is the persistence problem. A payload gets written into a memory store, a knowledge base, or a RAG corpus, and from that point forward, it fires every time the system retrieves it, not just once. The 2024 ChatGPT memory manipulation case showed this in practice: an injection persisted inside the memory feature and enabled data exfiltration across multiple separate conversations. That's a sustained, repeated exploit, a standing liability that outlives the session where it got planted.

Simon Willison's "lethal trifecta," coined in 2025, ties these together well. Nearly every serious documented attack combines three ingredients: an agent with access to private data, exposure to untrusted content, and a way to send information back out to the world. Take any one leg away, and the same injection mostly fizzles. That's why direct injection alone rarely produces catastrophic outcomes; it's the combination that turns a prompt into a real breach, which is exactly what the next section shows in documented production systems.

What prompt injection actually accomplished in documented production systems

EchoLeak is probably the clearest example on record. Disclosed in June 2025 and tracked as CVE-2025-32711 (CVSS 9.3), it hit Microsoft 365 Copilot. Aim Security researchers found it, and a paper by Pavan Reddy and Aditya Sanjay Gujral at George Washington University laid out the mechanics in detail. A single crafted email was enough. No click, no user action of any kind. Copilot read the email, followed instructions buried inside it, reached into internal files, and sent the contents to a server the attacker controlled.

The attack chained several bypasses together. It slipped past Microsoft's XPIA classifier, dodged link redaction using reference-style Markdown formatting, exploited how Copilot auto-fetches images, and abused a Microsoft Teams proxy that the content security policy happened to allow. Aim Labs called it an "LLM Scope Violation," meaning untrusted input from outside the system caused the model to step past a boundary it was supposed to respect. Whatever Copilot had access to was fair game: chat logs, OneDrive files, SharePoint content, Teams messages. Aim Security created a proof of concept and reported the issue in January 2025; a server-side fix was deployed in May and the advisory published in June, confirming no evidence of exploitation in the wild.

Salesforce Agentforce had its own version, called ForcedLeak (CVSS 9.4), disclosed by Noma Security in September 2025. A Web-to-Lead form served as the entry point. An attacker could plant instructions inside a lead record, and Agentforce would run them during ordinary employee use, no special access required. The exfiltration channel relied on a weak spot in a content security policy whitelist, including an expired domain that could be bought for five dollars. Salesforce rolled out Trusted URLs Enforcement for Agentforce and Einstein AI that September as a fix.

GitHub Copilot had a remote code execution flaw, CVE-2025-53773 (CVSS 7.8), patched in 2025. The injection exploited Copilot's ability to modify project configuration files. Security researchers demonstrated the attack was not a quirk tied to one model.

Devin, the agentic coding assistant, fared no better under scrutiny. Researchers testing its defenses described the agent as essentially defenseless: it could be manipulated into exposing ports to the open internet, leaking access tokens, and installing command-and-control malware, all through crafted prompts.

Slack AI widened its own attack surface in August 2024 when an update added file and Google Drive ingestion. A poisoned file, uploaded by any user, could now carry an injection payload straight into the system. Slack later shipped a patch addressing the issue.

In December 2025, Palo Alto Networks' Unit 42 reported a real-world indirect injection built to bypass an AI-based ad review system, layering multiple injection methods into a single payload. That detail matters. Attackers aren't just repeating the same trick anymore, they're combining methods and going after higher-value targets.

Even resume screening shows the pattern. A study tracking de-identified resumes from 2019 through the first half of 2025 found the fraction containing malicious content rising notably in 2024 before detection rates shifted by 2025-H2 as screening vendors started applying mitigations. Worth pausing on: this means the attack class isn't confined to security-forward companies running red-team exercises. It shows up in ordinary hiring pipelines too.

Look across every case above and a pattern emerges. None of these succeeded because someone left an obvious door open. They succeeded because normal, intended behavior, RAG retrieval, memory persistence, config file editing, form submission handling, became the delivery mechanism for the attack.

How agentic AI multiplies the blast radius of a single injection

A standalone chatbot that falls for an injection produces bad output. Embarrassing, maybe, but bounded, since it just talks. Hand that same model tool access and let it act on its own, and one injection can now trigger a chain: pull data, send it somewhere, modify a file, call another system. The output stops being words and starts being actions.

The blast radius scales directly with what the agent can reach. An agent wired into Salesforce, Microsoft 365, and Workday all at once doesn't expose one person's data if it's compromised. It exposes the combined authority of every permission across every one of those systems at once. Research has found that AI agents move far more data per session than human users do, which is exactly why a single compromised agent tends to be a high-magnitude event rather than an edge case.

Anthropic's own Claude Opus 4.5 system card ran indirect prompt-injection tests in agentic coding environments using the Gray Swan Shade tool. On the "thinking" variant, attack success climbed from 4.7% at a single attempt to 33.6% at ten attempts and 63.0% at a hundred. That progression is the real lesson here: persistence alone, no cleverer technique required, multiplies an attacker's odds even against one of the strongest models available. Multi-hop attacks, where an injection propagates from one agent or tool to the next, represent a growing concern as agentic deployments expand, which tracks: once one hop works, chaining a second and third is mostly a matter of patience.

A paper on what researchers call the "promptware kill chain" reframes prompt injection as the opening stage of a full malware-style intrusion, laid out across seven stages: initial access, privilege escalation, reconnaissance, persistence, command and control, lateral movement, and actions on objective. Under that framing, the injection itself is just the front door. Everything that follows looks like a conventional intrusion, just kicked off through natural language instead of a software exploit.

Cisco's State of AI Security report for 2026 found that 83% of organizations plan to deploy agentic AI, but only 29% feel ready to do it securely. That 54-point gap is where the amplification problem actually lands: in the space between how fast companies are shipping agents and how prepared they are to contain what happens when one gets manipulated.

It helps to think of a compromised agent as a machine version of an insider threat. It holds legitimate credentials. Its API calls look completely normal from the outside. But it's now running instructions an external attacker slipped in, and a firewall built to spot outsiders has no way to tell the difference. Ask yourself: what would your logs even show, if the compromised process was authorized to do everything it just did?

Where current defenses fall short and why no single control closes the gap

Start with where the field actually stands. No complete fix exists, not even for frontier models. OpenAI, Google, and Anthropic all remain vulnerable after applying their best available defenses. Nick Winter, VP of product and growth at Gray Swan AI, said in July 2025 that in these matchups, "typically [prompt injection] attack is winning against defense." That's a blunt assessment from someone building the tools meant to stop these attacks, and it's worth taking at face value.

Adoption numbers back that up in an uncomfortable way. Only about a third of organizations had deployed any dedicated prompt injection defense at all, meaning most enterprise AI deployments are running exposed. OWASP found that a large majority of production AI deployments it assessed in 2025 carried exploitable prompt injection vulnerabilities.

Why do the tools that work for other kinds of attacks fail here? Because prompt injection lives at the semantic layer, not the network or application layer. A web application firewall can inspect traffic patterns all day long and never notice a sentence that quietly tells a model to ignore its instructions. Input sanitization is built to strip out malformed code, not to catch a perfectly grammatical sentence doing something malicious.

The published defense numbers tell their own story. SecAlign, one of the stronger defenses on record, still misses roughly one in ten optimization-based attacks. Newer alignment-based defenses have pushed attack success rates down on some benchmarks, which sounds strong until you notice that figure comes from a static test, not from an attacker actively adapting to it. Give an adaptive attacker enough time to optimize against a known defense, and most published defenses fall.

Images are affected too. Researchers have demonstrated multimodal injections, instructions hidden inside images, that slip straight past filters built only to scan text. That raises real concern for fields like healthcare, where models increasingly analyze images alongside text, and where cross-modal attacks are genuinely hard to catch with today's tools.

OpenAI acknowledged in December 2025 that some injection attack vectors against AI browsing agents are likely a permanent feature of how these systems work, not a bug waiting on a patch. In response, OpenAI launched Lockdown Mode for ChatGPT Enterprise plans in February 2026, with a broader rollout to personal and self-serve accounts following that June. Read that as an admission: for some use cases, the answer isn't a smarter filter, it's restricting what the system is allowed to do in the first place.

One gap tends to get underestimated by security teams: visibility. Without runtime telemetry showing what an agent is actually doing step by step, defenders are stuck reviewing what the configuration says should happen, rather than watching what's actually happening. That's a meaningful difference, and it's where a lot of incidents go unnoticed until well after the fact.

Shadow AI makes all of this worse. IBM found in 2025 that breaches involving a shadow-AI component ran roughly $670,000 higher than breaches without one. Unvetted models and unofficial connectors sit entirely outside whatever governance a company thinks it has in place, and they widen the attack surface in ways nobody's tracking.

The layered defense architecture that production systems actually require

Given all of that, the only workable strategy is defense in depth. No single control closes the gap, so the architecture has to start from the assumption that some injection attempts will get through, and build for containment and detection, not just prevention.

Privilege minimization is the foundation everything else sits on, answering the amplification problem from earlier. An agent should hold only the permissions its specific task requires. An agent that can't touch SharePoint can't leak SharePoint content, no matter how convincing the instruction buried in an email happens to be. That means identity and access management needs to treat AI agents with the same seriousness applied to human employees: proper token management, and authorization policies that adjust based on task and context rather than standing access granted once and forgotten. Shrink what an agent can reach, and you shrink what a successful injection can do with it. It's the layer everything else depends on, not a nice-to-have layered on top of everything else.

Input and output controls form the next layer, and they matter, just not as much as people assume. Input validation and filtering catch known injection patterns before they ever reach the model, and this works reasonably well against direct injection, where the attacker is typing into a visible chat box. It's far weaker against indirect injection, where the malicious text is dressed up as an ordinary document, email, or database record and doesn't look like an attack at all on the way in. That's precisely why privilege limits and runtime monitoring have to sit alongside filtering rather than in place of it. Filtering catches what it recognizes, and indirect injection is built specifically to avoid recognition.