RAG System Data Leakage Risks

RAG is not a lookup. It is a multi-stage pipeline with distinct handoff points, and each stage introduces a different kind of exposure.

The pipeline moves in three phases. During ingestion, documents are chunked into fragments and converted into vector embeddings, then stored in a vector database. During retrieval, a user's query is itself embedded and used to find semantically similar document chunks. During generation, those retrieved chunks are inserted into the language model's context window alongside the user's prompt, and the model synthesizes a response.

To understand why this works, we must first look at two architectural properties underneath this process — the root conditions that almost every threat in this piece exploits.

First: when a document is converted to an embedding, its original access controls are stripped. The permissions, ACLs, and classification labels that governed the source document do not travel with the vector representation. The vector database holds semantic content, not the governance layer that protected it.

Second: retrieved content is passed to the language model as trusted context by default. Most RAG implementations do not verify whether retrieved content is legitimate before handing it to the model. The model receives it, and acts on it. No questions asked.

Traditional data exposure is usually a boundary failure. Someone gets past an authentication wall and reads data directly. RAG leakage can happen with the system working exactly as designed, surfacing sensitive material in model outputs to users who were never meant to see it. That difference matters more than most organizations have had time to reckon with.

How Retrieval Without Access Controls Surfaces Documents Users Were Never Meant to See

When a RAG system indexes a document corpus without preserving the original permission model, every authenticated user can potentially retrieve any document, regardless of their clearance level for the underlying source material.

Make it concrete: an executive compensation report, a legal hold memo, or a personnel file gets indexed alongside general company policy documents. An employee asking a routine HR question about leave balances receives a response that draws on material they would never have been permitted to read directly. The failure is structural, not incidental. Nobody made a mistake in the moment. The mistake was made weeks earlier, during indexing.

Without role-aware retrieval, a layer that actually filters results against the querying user's real permissions, the vector database becomes a flattened, permission-free copy of the organization's most sensitive knowledge. Hierarchies that existed in the source systems dissolve. Clearance levels carefully maintained in SharePoint or a document management system become irrelevant the moment those documents are ingested without their governance metadata.

This oversharing occurs through both internal-facing deployments, like employee assistants, and customer-facing channels. The exposure surface is determined by who can submit queries, not just who has document access. And that is a distinct threat model than most security teams are accustomed to thinking through.

It is also worth considering what makes this the leakage vector organizations most consistently underestimate: it requires no attacker. Only an improperly scoped index and an unsuspecting user asking a reasonable question. The system is malfunctioning in no way. It is doing exactly what it was built to do.

How Queries Themselves Leak Sensitive Information Before Any Document Is Returned

There is a leakage vector entirely separate from document exposure, and it operates upstream of any retrieval result.

The retrieval server sees every user query and its embedding. That query stream can contain sensitive personal, commercial, or operational information before a single document is touched. A clinician asking about a specific patient's drug interaction. A lawyer querying a deal structure that has not closed. An analyst asking about earnings figures that have not been released. All of this flows to the retrieval infrastructure the moment the user hits enter.

In enterprise RAG deployments using third-party vector database services or cloud-hosted retrieval infrastructure, this query stream may transit systems outside the organization's direct control. The organization may have carefully secured its knowledge base while the queries interrogating it pass through infrastructure it does not own.

The affected party here is the query sender, not the knowledge base owner. That is a compliance and privacy dimension separate from, and additional to, document leakage, and it is one that most RAG deployments were never designed to address.

If queries contain personal data, the retrieval server's exposure to them may constitute processing of personal data under GDPR and similar frameworks. The system is not just answering questions; it is, in the regulatory sense, processing the information embedded in those questions. That raises an important question: if the retrieval layer is processing personal data, which entity bears the compliance obligation — the organization that deployed the system, the vendor hosting the infrastructure, or both? That framing tends to surprise people the first time they encounter it.

Indirect Prompt Injection Through Retrieved Documents, and Why It Is Hard to Detect

OWASP ranks prompt injection as the top risk for LLM applications, and notes explicitly that RAG does not eliminate this risk. It creates a new entry point for it.

The mechanism: an attacker places a document containing hidden instructions into the knowledge base, through a public channel, a shared workspace, a web page the system's crawler indexes, or a poisoned file upload. A legitimate user submits a normal query. The RAG system retrieves the malicious document as a semantically relevant result. The hidden instructions are passed to the language model as trusted context, and the model follows them rather than the original system prompt.

Research presented at USENIX Security 2025 showed that as few as five carefully crafted documents targeting a specific query can manipulate AI responses with over 90% success, even against a database containing millions of documents. The signal-to-noise ratio the attacker needs is remarkably small.

Detection is hard, and not just because the problem is novel. The malicious document need not be the top-cited source in the response. The Slack AI incident demonstrated that the malicious message was not visible on the first page of search results, yet the injected instruction executed. The model's response bore no obvious trace of where the instruction originated.

There is also a priority override problem. When retrieved content conflicts with the system prompt, external retrieval content frequently overrides the original constraints. The model's deference to context is the vulnerability. It is doing what it was trained to do, which is to integrate provided context into its responses. But what if that deference is precisely what makes the system impossible to reliably constrain through prompt-level instructions alone?

The attack can also sit dormant in the index until a relevant query triggers it. Standard security scanning that evaluates documents at ingest time will miss an instruction that only becomes dangerous when paired with a specific future query. You scan it, it looks clean, and then six months later someone asks the right question.

What the Slack AI and Microsoft 365 Copilot Incidents Show About Production RAG Exposure

In August 2024, PromptArmor demonstrated exfiltration of private data through indirect prompt injection in Slack's RAG feature. The attack path was precise: a malicious prompt embedded in a public Slack channel post was ingested into Slack AI's knowledge base. A victim user's legitimate query retrieved it. The injected instruction executed and exfiltrated API keys from private channels the attacker could not directly access.

The attacker never needed access to the private channel. The RAG pipeline bridged the access boundary for them. Slack acknowledged and patched the vulnerability the same day, describing the attack as requiring very limited and specific circumstances. Those circumstances, as it turned out, described a normal enterprise Slack workspace.

Microsoft 365 Copilot saw a similar structural failure, documented by Aim Security in a vulnerability they named EchoLeak. A single crafted email caused Copilot to exfiltrate sensitive documents to an attacker-controlled URL with no user interaction required. Copilot's deep integration with Teams, SharePoint, and Outlook makes its retrieval surface broad; that breadth is also what made the exposure consequential.

Both incidents share the same structural signature: the attacker used the RAG system's own retrieval and generation pipeline as the exfiltration channel. Not a network breach, not a stolen credential. The designed behavior of the system became the attack surface. That is the shift that matters. The security model most organizations inherited was built around protecting data at its perimeter. RAG turns the data retrieval mechanism itself into the perimeter.

The upper bound of what indirect injection can achieve is illustrated by a SCADA case in which PDF attachments with invisible instructions, white text on a white background, caused an AI integrated with industrial control systems to execute commands that resulted in physical equipment damage. At that scale, the data leakage conversation becomes something else entirely.

These are not edge cases from controlled research environments. They are documented failures in products with Fortune 500 deployments, operating under real conditions.

How Attackers Reconstruct Knowledge Base Content Through Embeddings and Membership Inference

There is a persistent assumption among organizations that have deployed RAG: vector embeddings are not human-readable, therefore they are safe. This assumption is false, and the gap between perception and reality here is significant enough to be worth dwelling on.

Embeddings are not encrypted. Their content is recoverable. Research presented at ACL 2024 showed that attackers using surrogate embedding models can recover original document content from vectors without direct access to the embedding model, building on earlier work showing recovery rates for original input words in the range of 50 to 70 percent.

Recovery is not uniform across content types, and that non-uniformity is precisely the problem. Proper nouns, technical terms, and unique phrases occupy distinctive regions of the embedding space and are recovered at higher rates than generic language. This is exactly the content, product names, deal terms, patient identifiers, that organizations most need to protect. The math works against you in the cases that matter most.

University of Pisa research demonstrated that a fully black-box attacker with only query access can reconstruct the majority of a private RAG knowledge base through adaptive, relevance-guided querying. No direct database access required. The system answers questions, and the pattern of those answers reveals what it knows.

Membership inference attacks operate on a related but distinct principle. An adversary determines whether a specific document exists in the knowledge base by providing the first half of a document and requesting completion, prompting the model to predict masked tokens, or running multi-query interrogation designed to expose whether a document must be present to produce the observed responses. The value to an attacker is considerable: competitive intelligence, mapping of regulatory exposure, reconnaissance before a targeted extraction attempt.

OWASP formalized this concern by introducing Vector and Embedding Weaknesses as a dedicated security category, a signal that the field now recognizes the vector database as a primary attack surface rather than a safe intermediate store.

The policy implication follows directly: vector databases should be classified and protected with the same controls applied to the source documents they represent. Most current deployments do not operate this way. One might argue that this is simply an oversight that better tooling will eventually solve — but the structural incentives pushing organizations toward broad, lightly governed indexing have not changed, and tooling alone rarely corrects an incentive problem.

Knowledge Base Poisoning as a Supply-Chain Attack on RAG Outputs

Where indirect prompt injection arrives through a user's query, knowledge base poisoning arrives earlier. The attack vector is the data store itself. An adversary who can insert or modify documents in the knowledge base can pre-position malicious instructions that execute when a relevant query triggers retrieval.

PoisonedRAG, accepted to USENIX Security 2025, is the first formally validated knowledge corruption attack of this type. Semantically meaningful poisoned texts are injected into RAG databases to induce language models to generate attacker-controlled outputs. The outputs look coherent. They reference real context. They pass the casual read.

The dormancy property is what makes this category particularly resistant to conventional security practice. A poisoned document sits inert until the right query activates it, creating a window between ingest and execution during which standard scanning will not flag it as malicious. You cannot evaluate the document in isolation. Its danger is conditional on future context.

The attack surface for poisoning is broader than it initially appears. Any contributor to the knowledge base, including internal employees, external collaborators, third-party content feeds, and web crawlers indexing public pages, is a potential insertion point. Organizations using vendor-supplied RAG systems or third-party document connectors inherit the security posture of those upstream sources.

This is where RAG risk intersects directly with third-party vendor risk. When a RAG system ingests data from partner portals, SaaS integrations, or public web content, the organization has limited visibility into whether that content was tampered with before ingestion. The knowledge base is only as trustworthy as every data source feeding it, and in most enterprise deployments, that supply chain is long and only partially visible.

Where GDPR, the EU AI Act, and Audit Obligations Land for RAG Deployments

The European Data Protection Board's 2025 report on AI privacy risks specifically flags RAG systems as posing compliance risks related to data minimization, transparency, and lawful processing. That flag did not appear by accident, and it is worth understanding why these systems drew specific attention rather than being absorbed into general AI guidance.

The EU AI Act's Article 12 creates direct audit obligations for RAG systems operating in high-risk categories: HR decision support, credit risk assessment, healthcare diagnostics, education, critical infrastructure. Operators must maintain logs of system inputs and outputs sufficient to enable post-hoc auditing. The practical question regulators will ask is straightforward: which documents were retrieved, in response to which queries, and what controls governed that retrieval? Most current RAG deployments cannot answer those questions. Retrieval is opaque by default. The enforcement deadline is August 2, 2026, and the penalty structure is not abstract: violations in high-risk categories reach up to €15 million or 3% of global annual turnover; penalties for prohibited practices reach €35 million or 7%.

The data minimization problem is structural. RAG systems are typically built by indexing everything available, because that is the path of least resistance. GDPR's data minimization principle runs directly against this pattern, requiring that only data necessary for the specified purpose be processed. Nobody designed these systems to be non-compliant; they were designed to be useful, and compliance was a consideration for later.

The right to erasure creates a distinct operational challenge. If a data subject requests deletion of their personal data, the organization must identify and remove not just the source document but every chunk derived from it in the vector database. Most RAG deployments were never designed with that capability, because erasure was not part of the original specification.

The access control stripping described earlier is not only a security failure. It is a potential violation of purpose limitation: data processed under one legal basis becoming accessible for unintended purposes through the RAG interface. But how does this affect our original promise? The compliance failures and the security failures are, in many cases, the same failure observed from different angles — and organizations that treat them as separate workstreams risk solving neither completely.

Why Monitoring RAG Risk Requires Continuous Visibility Rather Than Pre-Deployment Review

Pre-deployment review is a useful starting point. It is not a sufficient endpoint, and treating it as one is a specific kind of organizational risk that is worth naming clearly.

The dormancy problem recurs across multiple threat types: poisoned documents, injected instructions, and over-indexed sensitive data all sit undetected until a query activates them. A static audit evaluates the knowledge base as it existed at a point in time. It cannot catch what is added or modified after the review closes.

Knowledge bases are not static. Documents are added, updated, and connected to new data sources continuously. Each change is a potential new exposure that was absent during initial deployment review. Organizations that assess RAG security once and move on are operating on an assumption of stasis that the actual system does not share.

Third-party vendor RAG deployments add another layer of complexity. Microsoft 365 Copilot and similar deeply integrated productivity tools update on vendor-controlled release cycles. New retrieval features, new data connectors, new integrations can alter the exposure surface without any action on the deploying organization's part. Waiting for the next security review cycle to catch those changes is not a defensible posture.

Query behavior over time also provides signal that point-in-time assessment cannot. Patterns that suggest membership inference probing, unusual retrieval of sensitive document categories by lower-privileged users, responses that appear to reflect content that should have been excluded from the index: these anomalies become visible across a query stream. They are invisible in a pre-deployment snapshot.

The organizations managing RAG risk most effectively are not the ones that completed the most thorough initial audit. They are the ones treating the deployed system as a continuously evolving attack surface and building observability accordingly. The question worth asking is not whether your RAG deployment was secure at launch. It is whether it is secure now, and whether you would know if something had changed since you last looked.