Guard LLM Approaches to Prompt Injection Detection

Prompt injection has held the top spot in OWASP's Top 10 for LLM Applications for two editions running, first flagged in 2023 and reconfirmed in late 2024 as agentic AI use grew. Guard LLMs, purpose-built classifiers that sit alongside a primary model and screen its inputs for injection attempts, are the industry's current answer to that threat. They're a real step forward from earlier defenses, but the research on how they fail is just as important as the research on how they work.

A language model still can't tell whether a system prompt was written by a developer or a user prompt was written by anyone with access to the chat box, and that hasn't changed since 2023. Both arrive as text in the same context window. Models also tend to weight the most recent instruction most heavily, a habit that's useful during training and a liability the moment an attacker figures out where the context window ends. None of this requires readable text, either. Any content the model parses, hidden Unicode, a webpage's alt text, a PDF's metadata, can carry a payload. Two attack classes exist: direct injection, where the override sits right in the user's message, and indirect injection, where it's smuggled in through a RAG pipeline, a scraped webpage, or a tool's output. Add one more wrinkle: an attacker can just keep trying. LLMs are nondeterministic, so a payload that fails nine times might land on the tenth, and that asymmetry shapes everything a defender has to build against.

What guard LLMs are and how they differ from earlier detection approaches

Detection methods sort into three families, roughly by how much engineering went into them.

The simplest is prompting-based and training-free: ask a second LLM to look at the input and flag anything suspicious. Known-answer detection checks whether the primary model still follows a hidden, pre-set answer after seeing the input. Spotlighting uses delimiters to wall off context from instruction. Both are cheap to set up and both inherit a weakness: they're built on the same kind of model they're trying to police, so whatever tricks fool the primary model tend to fool the detector too.

Fine-tuned classifiers, the guard LLMs proper, are the more serious answer. Here, a model (often a compact encoder architecture) gets fine-tuned specifically to output "injection" or "not injection." Because it's a separate model with its own training run, it doesn't automatically share the primary model's blind spots. This is the current state of the art, and it's where most of the commercial and open-source activity sits.

A third family, classical ML, covers things like fine-tuned variants of another pretrained language model and random-forest classifiers (Nvidia's NeMo Guard, for instance, uses pre-trained embedding pairs). These appear as one layer inside a larger guardrail stack rather than as standalone products.

One distinction gets missed constantly: general-purpose safety guardrails like GPT-OSS-Safeguard or Nemotron-Safety-Guard let you write custom detection prompts, but they're built to catch harmful or unsafe content, not prompt injection specifically. Deploying one of those and assuming it covers injection is a category error that costs something before it's caught.

The current field of named guard models and what each is built to do

Meta's Prompt Guard (2024) is fine-tuned from mDeBERTa-v3-base and works within a 512-token context window. Version 1 is a multilabel classifier catching both jailbreaks and injections; version 2 narrows to binary classification. Meta's own documentation recommends splitting longer inputs into chunks and scanning them in parallel, since 512 tokens doesn't stretch far. Meta also frames it explicitly as a starting point, not a finished product.

ProtectAI released two open-source models: v1 in November 2023, v2 in April 2024. Both are fine-tuned from DeBERTa-v3-base, which is roughly double the size of Meta's model. V2 made a deliberate scope cut: it's not trained to catch jailbreaks at all, only prompt injection.

Deepset's deberta-v3-base-injection takes Microsoft's DeBERTa-v3-base and fine-tunes it for a simple binary call: "INJECTION" or "LEGIT."

Lakera Guard is closed-source and commercial, built for real-time, low-latency screening. It pairs a proprietary threat database with its own models. Its closed-source nature makes independent benchmark comparisons difficult, and its performance trade-offs remain less publicly documented than open-source alternatives.

Hyperion, from Epivolis, runs on a RoBERTa backbone with a substantially larger parameter count and goes after both jailbreaks and injections, positioned as lighter-weight than a full LLM deployment.

Vijil's Prompt Injection model is a binary classifier fine-tuned from ModernBERT, aimed specifically at injections trying to provoke harmful responses.

InjecGuard was built to directly tackle over-defense, balancing three separate metrics: how well it catches malicious inputs, how well it lets benign inputs through, and how well it avoids over-blocking.

JavelinGuard (2025) spans five architectures of increasing complexity, Sharanga as the baseline, Mahendra adding attention-weighted pooling, Vaishnava and Ashwina using hybrid neural ensembles, and Raudra as a multi-task framework. It's built to run fast on standard CPUs and, notably, to distinguish jailbreak attempts from injection attempts, a separation the paper's authors flag as missing from most existing tools.

Across nearly all of these, the open-source field leans heavily on DeBERTa-family foundations. That shared ancestry matters later, because an evasion trick that works against one classifier built on that shared foundation has a real shot at transferring to the next.

How evasion attacks break guard LLMs in practice

A study out of Lancaster University (Hackett et al.) tested six prominent guard systems, including Microsoft's Azure Prompt Shield and Meta's Prompt Guard, and broke them with two distinct techniques.

Character injection inserts Unicode or special characters that a human reader barely notices but that scramble the model's tokenization enough to dodge classification. Adversarial Machine Learning evasion goes subtler still: tiny, near-imperceptible perturbations that exploit the classifier's over-reliance on specific learned features and its lack of exposure to varied training examples. Both approaches kept the underlying malicious prompt fully functional. The attack still works. The guardrail just stops seeing it. In some tested setups, evasion succeeded 100% of the time.

There's a second layer to this that raises the stakes for commercial products specifically. Evasion techniques developed against open-source, white-box models can transfer to closed-source, black-box targets. So even a proprietary detector that never leaks its own weights isn't insulated, if an open-source cousin exists somewhere for attackers to probe first.

What does this say about the underlying architecture? Any classifier trained to recognize a fixed distribution of known injection patterns is, by definition, learning surface features, not the deeper structure of manipulation. And surface features are what an adversary with access to a similar model can reverse-engineer and route around. Combine that with attacker persistence (the ability to just keep trying against a nondeterministic system) and evasion stops looking like a bug to patch and starts looking like a property of the approach.

What over-defense costs in practice

Catching every attack is easy if you're willing to block everything. The harder number is over-defense accuracy: how often a guard model lets a benign request through instead of choking on it.

Benchmark research found several open-source models struggling significantly on over-defense accuracy, where 50% is just a coin flip. That's a rough place to sit for a security control.

A benchmark called CAPTURE, built by researchers at Pure Storage and presented at the LLMSec workshop, dug into why. Testing context-aware attacks, meaning prompts where the malicious intent depends on surrounding context rather than an obvious trigger phrase, Fmops missed 100% of them (a 100% False Negative Rate). Testing benign inputs that happened to share structural similarity with attack patterns, PromptGuard flagged all of them as malicious (a 100% False Positive Rate). The researchers trace this to how these models learn: they latch onto trigger words from their training data, so anything that looks structurally like a known attack gets blocked, and anything contextually malicious but structurally novel slides through. Most existing benchmarks make this worse by testing prompts in isolation, stripped of the application context that real deployments always have.

A newer evaluation called PIArena (2026) surfaces a sharper version of the same failure. When an injected task and the legitimate target task point in a similar direction, telling them apart becomes genuinely ambiguous, not just hard for current tooling, but conceptually unclear even to a careful human reviewer. That's a defense making things measurably worse.

The opposite failure mode is just as telling. Some detection methods, tuned for maximum safety, can push utility so low that they stop being useful as a product.

Every production system built on top of one of these classifiers eventually runs into the same operational squeeze: false positives block real users, users complain, someone lowers the detection threshold to reduce friction, and that lowered threshold is exactly the gap an evasion attack walks through. The available benchmarks show no tested system handles both halves of the trade-off at once. Nothing tested so far handles both halves of the trade-off at once.

What research-stage methods reveal about structural limits in current detection

Three newer approaches try to escape the training-data trap in different ways, and each one exposes a different limit.

AttentionTracker (2024-2025) skips classification entirely and instead watches where the model's attention goes, on the theory that an injected instruction visibly pulls focus away from the original task. Among training-free methods, it improved average AUROC by 31.3% on the Open-Prompt-Injection benchmark and 20.9% on the deepset dataset. Measured against every method including trained classifiers, it still added up to 3.1% on Open-Prompt-Injection and up to 10.0% on deepset. Strong numbers, but circle back to the earlier point: this is the same method that zeroes out utility on most datasets when tuned for maximum safety. Open-Prompt-Injection and up to 10.0% on deepset show the signal, but circle back to the earlier point: this is the same method that zeroes out utility on most datasets when tuned for maximum safety. The trade-off is brutal.

PIShield (2025) takes yet another angle, reading the model's internal residual-stream representations and feeding them into a simple linear classifier, no fine-tuning of the target model needed. It's fast, 0.033 seconds per sample, on par with lightweight classifier methods, and it holds a low false-positive rate on long-context datasets like Musique and NarrativeQA, exactly where other detection approaches struggle.

That gap points at something specific: long context is where these methods break. DataSentinel (2025) confirms it from the other direction. It uses a minimax, game-theoretic setup, fine-tuning a detector against adversarially adapted payloads, and it holds utility well on short-context datasets. But push it into long-context settings and utility falls to 55% on average. Training a model to resist evasion, in other words, can directly cost it usability once the input gets long and complicated.

Line these three up: two structural findings hold across all of them. First, context length degrades every current approach, not just one architecture, one training method. Second, none of these systems produces an interpretable rule for what actually counts as injection. The decision comes out of a black box, which is a real problem for any enterprise security or compliance team that needs to explain why a request got blocked.

Why committing to a single detector is the wrong architecture

Every guard model reviewed here is strong somewhere and weak somewhere else. None dominates across attack types, context lengths, and application domains at once. PIArena's 2026 findings make the point concretely: defenses that scored well on the benchmarks they were built for showed real trouble holding up against diverse, adaptive attacks. That's not one product failing to live up to its marketing; it's the whole category running into the same wall. That's the whole category running into the same wall.

So what's the fix, if no single detector gets there? A newer framework called SCOUT (2026) reframes the question entirely: instead of picking one detector and trusting it, treat detection as an allocation problem. Given a mix of detectors with different strengths, decide per request which ones to run and when to escalate to a full LLM judge. SCOUT predicts each detector's likely reliability and latency on a given input by comparing it to similar past inputs, and it exposes a single tunable safety-utility threshold to whoever's operating it. Tested on a purpose-built benchmark of structurally complex, agent-facing injections (SCOUT-450) plus three external benchmarks, BIPIA, IPI, and IHEval, it beat every single-detector baseline on the safety-utility trade-off, including a setup that ran an LLM judge on every single request.

Separately, multi-agent defense pipelines, specialized LLM agents working in coordination for real-time detection, are being explored as another path toward the same goal. The researchers behind that work call it a promising foundation, while flagging real open problems: adaptive adversaries, multi-turn attacks, and the resource cost of running several agents per request.

Ensemble and adaptive architectures aren't just redundancy for redundancy's sake. Ensemble and adaptive architectures aren't just redundancy for redundancy's sake. They're a response to a structural mismatch: any fixed classifier has a training distribution, and any determined adversary can probe that distribution and shift the attack to sit outside it.

What enterprises must assess before deploying guard LLMs as a primary control

The mismatch between generic training data and a specific application is the starting point. Guard models trained on broad, public datasets generalize poorly to whatever a particular product actually looks like in production. That is why Meta's own Prompt Guard documentation tells adopters to fine-tune on their own application data before shipping it. Skipping that step means running a detector calibrated for someone else's traffic patterns.

Context length deserves its own line item: it is consistently a failure point across AttentionTracker, DataSentinel, and PIShield alike. Any team evaluating a guard model should ask, specifically, how it performs on long documents, multi-turn conversations, and RAG pipelines pulling in outside content.

The evasion research points to a broader gap that outlives any single product update: enterprises need visibility not just into how a detector scores on a benchmark, but into how it changes over time. Vendors patch models, retrain classifiers, and quietly shift behavior, and a detector that held up in last quarter's testing can fail silently against a new evasion technique without anyone noticing until an attack gets through. Promptarmor's work in this space focuses on that gap: continuous monitoring of how AI vendor tools evolve and where new vulnerabilities occur, aimed at catching an evasion problem before it turns into a production incident rather than after.

None of this argues against guard LLMs. It argues against betting an entire security posture on one of them, tuned once and left alone. The research consistently points toward layered, adaptive setups, monitored over time, rather than a single classifier asked to hold the line indefinitely against an adversary that only has to find one gap and can try as many times as it wants.