Scoring RAG Retrieved Chunks for Instruction-Class Content Before AI Agents Act on Them

To protect an enterprise RAG pipeline from indirect prompt injection, retrieved chunks need to be scored for instruction-class content before an agent acts on them. Relevance checks measure whether a document helps answer a query, but they do not check whether it tries to command the agent. A separate, query-aware scoring gate addresses that gap, while provenance controls and continuous monitoring cover risks that a single gate cannot.

Why retrieved chunks are an untrusted input surface, not passive context

Retrieval can look safe because it is read-only. The agent fetches a document rather than changing it, so the operation appears to leave little room for harm. But the retrieved text becomes part of the model’s prompt, where it can influence behavior. Reading is not the same as trusting. A chunk crosses a trust boundary when content from an outside source enters the context the agent uses to decide what to do.

Classic RAG makes this easy to overlook because its flow is simple, with one query, one retrieval and one generated answer. That prompt construction does not itself give the model a reliable way to separate data from instructions. Agentic RAG gives the agent more chances to confuse data with instructions because it decides whether to retrieve, what to retrieve and whether to retrieve again. The user’s query can stay harmless the whole time. Documents, wiki pages, vendor uploads and indexed Slack threads are a separate input surface that user-prompt checks do not cover.

How indirect prompt injection travels from a stored document to agent action

An attacker can plant imperative language in a document that enters the retrieval corpus. A semantically related query then brings the poisoned chunk into the model’s context together with legitimate information. If the model treats the document’s words as commands rather than material to examine, the attacker’s text can steer tool calls, messages, record creation or downstream automation.

Microsoft 365 Copilot illustrates the full retrieval-to-action risk. In a documented zero-click indirect prompt injection, an attacker sent one crafted email, and the user did nothing. Copilot processed the message in context and could be made to pull internal files and exfiltrate their contents to an attacker-controlled destination. The compromise did not require a malicious user query. Devin AI provides a corroborating case, with security researchers finding almost no protection against prompt injection.

An indexed chunk remains available to queries that retrieve it and to agents sharing the index for as long as the document remains. Its influence can also spread through response caches, stored summaries, persistent agent memory or downstream indexes built from agent outputs. Text is not the only entry point. Hidden prompts in images, OCR output, captions and file metadata can feed the same path into the model’s context.

What relevance scoring catches and what it structurally cannot

Relevance scoring answers a useful but separate question. It measures semantic proximity to the query, removes off-topic chunks and improves the grounding of an answer. None of those tasks classifies a chunk’s behavioral intent. A document can be highly relevant while also telling the agent to “ignore all previous instructions” or “use the tool.” Its topical fit can help the attack reach the model.

CRAG (Corrective RAG) establishes a precedent for scoring between retrieval and generation. Its relevance classifier scores every chunk, and when results fall below the threshold, a web search adds to what internal retrieval found before generation. MAIN-RAG uses multiple LLM agents to filter and score documents collaboratively, with an adaptive threshold based on the score distribution. Both assess retrieval quality. Their scoring architecture does not turn relevance into an instruction detector.

A relevance score cannot reveal a threat that is independent of topical similarity, just as a high-quality answer source is not automatically an authorized source of commands. A 2026 layered-architecture paper distinguishes instruction-class detection from relevance filtering as addressing a different stage of the attack lifecycle. Hallucination guardrails also occupy the post-retrieval boundary in the 2026 production RAG stack. The architectural space for another check exists, but the check needs its own signal.

What instruction-class scoring means and what signal it surfaces

Instruction-class scoring checks whether retrieved content tries to tell the agent what to do. Its targets include imperative language, role overrides and claims of authority expressed through phrases such as “act as,” “system prompt,” “developer note” and “send the following.” A tutorial can contain them legitimately, while an attack can express the same intent without using any familiar keyword.

The assessment therefore draws on three distinct signals. The first is instruction-likeness in the raw chunk, which can flag obvious directive language before prompt assembly. The second is semantic classification of the joint query and chunk, which asks whether the retrieved text, in this context, could be treated as an unauthorized command. The third is the source’s provenance trust tier, preserved from ingestion through retrieval to prompt assembly. Together, these signals distinguish what the text says, how it relates to the request and where it came from.

Harmful behavioral intent is not the same as objectionable wording, and ordinary instructional language is not necessarily hostile. The 2026 MDPI study on joint embedding classification reports F1-scores above 0.85 and accuracy above 84% across configurations, which supports this approach but does not show that it catches every attack. Its classifier encodes the query and chunk into a dense vector, then predicts benign or malicious content with a confidence score. A chunk can score highly for relevance but also receive an instruction-class warning.

Where in the RAG pipeline the scoring stage fits and how each signal is applied

The scoring gate belongs after relevance reranking and before the top-k chunks enter the prompt. This preserves the relevance order while adding a separate behavioral assessment. At ingestion, lightweight instruction-likeness scoring can add risk annotations or allow optional pre-filtering without adding work to the query path. That early pass catches obvious patterns. The query-aware assessment then runs at retrieval time, when the actual request is available, with batching addressing the latency constraint of scoring multiple chunks.

nokast-secureRAG, described in an October 2026 arXiv paper, uses a small language model as a semantic firewall between retrieval and generation. It jointly evaluates the query and retrieved context, classifying the pair as safe, suspicious or malicious. On a held-out in-distribution test set, it achieves 0.994 detection recall at a 0.026 false-positive rate. Those conditions qualify the result. They show how well it worked on that test set, but do not guarantee the same results across every enterprise corpus.

RAG-PIBench, described in an October 2026 arXiv paper, provides another view of classifier options through 4,876 contextual examples across frozen train, validation and protected-test splits. DistilBERT achieves its best protected-test performance, with F1 = 0.896 and PR-AUC = 0.968, while TF-IDF SVM and logistic regression remain competitive. Provenance also helps weight risk scores or set thresholds based on source trust, so the gate treats suspicious content from a low-trust source with more caution. The gate can suppress a chunk, pass it with a risk annotation directing the model to treat it as untrusted data, or route it to human review for high-stakes agentic actions.

Why chunk-only classifiers fail and what the query-context joint approach corrects

High recall alone does not make a usable gate. Prompt-Guard has 0.900 recall when used on retrieved chunks, but its 0.690 false-positive rate means it flags most legitimate content. Such a gate can remove the very information retrieval was meant to supply. Documentation, policies, training materials and API references all have valid reasons to include commands. Without the query, a classifier lacks the context needed to distinguish instructions being discussed from instructions trying to redirect the agent.

The joint approach addresses that missing relationship. The firewall model’s ablations show that retrieved context provides the main attack signal, while query-context consistency reduces false positives. Its joint model’s false-positive rate is 0.026, compared with 0.690 for the context-blind baseline. Assessing directive language requires determining whether it is appropriate to this request.

This also clarifies the role of ingestion-time checks. They can annotate suspicious chunks, but index-time detection alone cannot replace the final assessment made when the query is known. In multi-hop agentic RAG with up to six retrieval calls per turn, false positives compound across hops. Even moderate suppression at each stage may leave the agent with insufficient context by hop three or four.

What scoring cannot do and where it fits in a layered defense

A retrieval gate controls one route into the agent. It does not automatically check tool responses or API payloads that arrive later, and it does not cover messages passed between agents. MCP tool descriptions and metadata can contain adversarial instructions without entering the RAG path at all. Email and documents sent directly for summarization can bypass the indexed corpus too.

OWASP states that the stochastic nature of model behavior rules out foolproof prevention and that no single tool reliably detects every prompt injection attempt. The layered-architecture paper likewise treats its defenses as nonredundant. Layer 1 prevents injections from entering the pipeline, while Layer 2 constrains which instructions the model may follow based on provenance when an injection gets through. Filtering and authority limits do different work. Combining them into one detection gate leaves the second function unaddressed.

OWASP Top 10 for Agentic Applications, published December 2025 and labeled 2026, identifies Agent Goal Hijack (ASI01) as the top threat, often executed through indirect prompt injection. An OWASP GenAI Q1 2026 report on exploits consolidates eight major incidents from January 1 through April 11, 2026, mapping each to the LLM Top 10 and Agentic Top 10. These frameworks place the problem in operational remediation.

Continuous monitoring of retrieval behavior as the control that scoring alone cannot provide

A scoring decision describes content at one moment. Internal wikis, Confluence, Notion, Google Drive, code repositories and support tickets change continuously, so a clean document can become poisoned at the next crawl. A sudden surge of new content from one source, or an abrupt shift toward instructional language, can indicate a change even when individual chunks do not cross a detection threshold.

Observing which tools the agent calls, which payloads it constructs and what it sends or stores can expose compromises that passed the gate, including paraphrased injections that evade classification. Monitoring re-ingestion also covers a persistent feedback path. Summaries that agents generate, memory stores and downstream embeddings can bring attacker-influenced material back into the index. Scoring only the original retrieved document leaves that secondary propagation outside view.

Plugins, connectors, MCP tools and third-party retrieval sources give outside content paths to agent action, often with less visibility into those paths than inside the enterprise pipeline. Their assessment and monitoring therefore need to operate at the vendor level, before and after deployment. Platforms that use both retrieval-layer scoring and continuous behavioral monitoring cover what static controls miss, especially for enterprise deployments and law firms with sensitive corpus data. The threat runs continuously, so the defense must too.