RAG System Data Leakage Risks

Retrieval-augmented generation won out as the default way enterprises deploy large language models because it solves a specific problem cheaply: it lets a model answer questions about current, company-specific information without the cost of retraining it. At inference time, a user's query gets converted into an embedding, that embedding gets matched against a vector database of chunked documents by cosine similarity, and whatever chunks come back closest get handed to the model as context alongside the original question. The model answers only with what the retriever found, nothing more and nothing less.

This pattern isn't confined to standalone chatbots anymore. LLMs sit inside IDEs, CRMs, ticketing systems, collaboration tools, and office suites, reading from the same inboxes, wikis, and databases employees use every day. And the knowledge bases feeding these systems aren't generic test data. Organizations build RAG over internal documentation, customer records, financial data, legal files, and source code, the material a company would least want exposed.

The same design choice that makes RAG useful, grounding answers in real proprietary context, is what turns a retrieval failure into a data breach rather than a wrong answer. A chatbot that hallucinates a bad fact is an accuracy problem. A chatbot that retrieves and repeats a real line from a confidential contract is a disclosure. Most teams assume access control at the retrieval layer already protects against this, but that assumption is where the risk actually lives.

The access-control gap that the retrieval layer does not inherit

Permissions set at the application layer, who can see which SharePoint folder, which Confluence space, which CRM record, do not automatically travel down into the vector database. The retriever has no built-in concept of user identity or document sensitivity. It converts a query to an embedding, finds the chunks with the highest cosine similarity, and returns them. That's the entire decision process. If a document was restricted to one department, the vector representing a paragraph of it carries no trace of that restriction once it's sitting in the index.

The practical result: a RAG system that leaks information leaks real text, a line from a private contract, a snippet of source code, a fragment of someone's personal data, simply because a query happened to land close to it in vector space. Multi-tenant SaaS products make this worse. When several customers or teams share one knowledge base with different access levels, the system has to enforce access control at the document level, otherwise no access control is enforced. Controlled experiments on multi-tenant RAG systems without explicit authorization checks found cross-tenant data leaking in 98 to 100 percent of probes. A missing check is close to a guarantee of privilege escalation, not a rare edge case.

Why does this gap persist when the fix sounds so straightforward? Cost. Maintaining a separate vector index per tenant is expensive to operate at scale, so most SaaS AI platforms default to logical isolation (shared infrastructure, separated by metadata tags) rather than physical isolation (separate infrastructure per tenant). Logical isolation works only as well as the code that enforces it, and you'll find that code outside the retrieval mechanism itself. Because RAG touches ingestion, retrieval, storage, and logging as distinct components, a single unchecked gap in any one of them can cascade into cross-tenant disclosure, and under GDPR, HIPAA, or CCPA that cascade becomes a regulatory incident, not just a technical one.

Enforcing document-level role-based access control at the retrieval layer is the correct response: most deployments don't do it by default. But even a team that implements it correctly has closed only one of three structural gaps. The next one lives somewhere access control can't reach at all: inside the vector representation itself.

Why embeddings are not a privacy boundary

A common assumption treats embeddings as a kind of safe abstraction, numbers that represent meaning without exposing the underlying text. That assumption doesn't hold up. Sentence embeddings encode enough semantic structure that standard inversion techniques can recover a substantial share of the original input words from a compromised vector. OWASP formalized this as its own risk category, LLM08:2025, Vector and Embedding Weaknesses, specifically because embedding-based retrieval is now central to how production models get grounded.

What does that mean in practice? A breach of the vector database is a breach of the documents those numbers were derived from, recoverable well enough to matter.

Consider who's most exposed by this. A healthcare provider embedding patient records, a bank embedding transaction narratives, a law firm embedding privileged client communications: for each of these, an inverted embedding is the same regulatory event as losing the source document, because enough of the source document comes back out.

Compounding this, vector database security tooling hasn't caught up. The field hasn't matured to match the sensitivity of what it stores, so even if your organization treats its vector store with care, you often can't detect or stop inversion attempts against it. Two structural gaps are now on the table: access control that never reaches the retriever, and embeddings that never functioned as a privacy boundary. The third gap is what connects these into one attack surface instead of two separate weaknesses, and it sits inside the model's own usefulness.

How the LLM's instruction-following capability becomes a leakage amplifier

Stronger instruction-following in the LLM correlates with higher leakage risk, a finding that should unsettle anyone treating RAG security as a configuration checklist.

A benchmark called LeakDojo (Zhang et al., ACL 2026) tested six attack types across multiple LLMs and four datasets, and you need to hold three findings together. Stronger instruction-following capability correlates with higher leakage risk. Improvements in RAG faithfulness, the very metric teams optimize for, can introduce increased leakage risk alongside it. And larger models carry a higher risk of disclosing context than smaller ones do.

Why would better models leak more? Instruction tuning, the fine-tuning process that makes a model responsive, helpful, and faithful to whatever context it's given, also makes it more susceptible to following malicious instructions buried in that context. A model good at doing what retrieved text tells it to do easily confuses text that's informing it with text that's commanding it. It's a direct consequence of the capability enterprises are paying for, not a flaw introduced by sloppy engineering.

LeakDojo's data breaks the problem into two components, query generation and adversarial instruction crafting, and finds that overall leakage approximates their product. Both pieces have to be addressed. Fixing only one leaves the other fully intact. And the procurement implication follows directly: choosing a frontier model for a RAG deployment is not a security-neutral decision. The LeakDojo results show capability gains across the model leaderboard tracking alongside measurable increases in leakage probability. You can't tune your way out of this trade-off with better prompts or stricter system messages. The fix has to be structural isolation between what the model is told and what it's allowed to act on, because the trade-off is mechanical.

How indirect prompt injection weaponizes the knowledge base itself

Put the three gaps together, broken retrieval-layer access control, invertible embeddings, and instruction-following that can't distinguish data from commands, and a single attack class emerges that exploits all three at once: indirect prompt injection through the knowledge base. The model can't reliably tell legitimate retrieved content apart from adversarial content planted inside a document, so if you have write access to any indexed source, you can turn that knowledge base into an injection vector against every user who later queries it.

The mechanism is patient by design. An attacker inserts or edits a document somewhere in the indexed corpus, embedding instructions the model will follow once that document gets retrieved. Nothing happens until a relevant query pulls the poisoned document into context, so the attack can sit dormant for weeks before it fires.

Researchers at Pennsylvania State University and Illinois Institute of Technology demonstrated this directly: a single poisoned document inserted into a RAG knowledge base reliably caused the model to follow embedded instructions, generating phishing content, leaking other documents, or returning dangerously wrong information, against multiple production models including GPT-4, Claude, and Gemini.

The pattern appears outside research settings too. In one documented incident involving Cursor IDE, attackers hid malicious text inside public GitHub README files. When developers opened or cloned those repositories, Cursor's AI assistant read the poisoned README straight into its context and carried out the hidden commands, which gave attackers unauthorized access to developer machines. That's injection through source code documentation, a part of the stack most teams never think to treat as untrusted.

The supply-chain version of this problem extends the risk further. Any organization that indexes third-party vendor documentation into its internal RAG system has created an indirect path into its own corpus. An attacker who compromises the vendor's docs gains a foothold in the target's RAG system without ever touching the target directly.

Open-source RAG infrastructure introduces vulnerabilities that enterprise governance doesn't yet cover

Everything above concerns the model and the retrieval logic. Both also depend on a separate layer, the RAG engine itself, often open source, often self-hosted, and running code that enterprise governance frameworks rarely account for.

Standard dependency scanning tools catch known vulnerabilities in application libraries. They have no visibility into whether the RAG engine is exposing a code-execution surface to its own authenticated users, a different category of risk that sits below what application security tooling was built to see.

This compounds the immaturity already raised with vector databases. Vector database security tooling is young. Open-source RAG engines, as a category, are younger still; the CVE catalog tied to this layer is expanding as adoption grows rather than shrinking as the tools mature. If your team treats RAG security purely as a prompt-engineering problem or an access-control problem, you miss this layer. RAGFlow's documented CVEs show that an authenticated low-privilege user, not an external attacker with special access, is sufficient to exploit them. Knowing about this layer is only useful if the response to it matches its urgency, and that's where enterprise practice is currently falling short.

Enterprise defenses lagging behind documented structural risks

Putting all four gaps together, retrieval-layer access control, embedding inversion, instruction-following as a leakage amplifier, and infrastructure-level CVEs, shows that most enterprises running RAG have not built defenses that match what the architecture actually exposes them to. The gap is not static. RAG adoption is scaling faster than the security practice built to govern it, so the distance between risk and readiness is widening rather than closing.

Splunk's 2026 AI Security Report puts a number on part of this: only a small minority of teams run a dedicated adversarial testing loop against their RAG pipelines. Most of what passes for defense today relies on heuristic filtering or incremental improvements to retriever robustness, both of which share the same weaknesses: limited interpretability, no formal security guarantees, and real vulnerability to attackers who adapt once they learn what the filter catches.

A 2026 survey cataloging the RAG security literature (arXiv:2604.08304) organizes the field across six workflow stages, three trust boundaries, and four primary security surfaces, and its conclusion is blunt: current defenses remain largely reactive and fragmented. The field now has a taxonomy. It does not yet have a defense.

Closing that gap starts with visibility that most teams currently lack. Organizations deploying RAG often have no clear answer to whether their chosen LLM vendor or third-party AI platform actually enforces document-level access control at the retrieval layer, a question that becomes especially hard to answer inside multi-tenant SaaS products where isolation is logical rather than physical. Continuous monitoring platforms, Promptarmor among them, are built to check specifically for this: whether retrieval-layer RBAC is actually implemented in practice, and whether gaps in data isolation exist before they cascade into cross-tenant disclosure or a regulatory violation.

The architecture of RAG created four distinct places for data to leak: the retriever, the embedding, the model's own instruction-following, and the infrastructure underneath all three. A defense strategy that addresses only one of them addresses a quarter of the actual exposure.