Cross-Session Query Correlation as an Early Signal of Incremental Context Extraction

Cross-session query correlation is becoming an early-warning signal for system prompt confidentiality and data exfiltration risk because it links requests that session-bound guardrails judge one at a time. Detecting incremental context extraction requires persistent interaction records, intent-aware retrieval, and evidence of what the model actually produced. The aim is to recognize a harmful trajectory before its fragments become a useful whole while distinguishing an attack from ordinary related queries.

Why session-bound guardrails cannot see the attack that spans them

A harmful objective can be spread across a dozen individually harmless sessions. Each guardrail sees a request and response that it can allow, while the attacker keeps the pieces from every interaction. Nothing looks dangerous on its own. The gap lies between the boundary used to judge safety and the boundary within which the attacker works.

Natalie Isak and Matthew Dressman’s Magnet describes this asymmetry directly. An agent’s context window is refreshed between sessions, but the person or system driving it retains what came before. The same study says most state-of-the-art AI abuse detection focuses on single-turn attacks or multiple turns within one session.

What changes when agents also share context? Crew Scaler’s response to NIST RFI 2026-00206 describes systems that share persistent memory, propagate tool authorization through delegation chains, and influence one another’s reasoning. Existing security and governance frameworks were not designed for those relationships. The security surface emerges from behavior across boundaries, rather than remaining inside a single prompt that a classifier can inspect.

How cross-session decomposition attacks are structured in practice

Cross-session decomposition separates the collection of answers from their eventual use. An attacker splits a forbidden objective into benign-looking subqueries, asks them in independent interactions, and combines the responses outside the model. The forbidden objective need never appear in a request. That makes the relationship among answers more revealing than the wording of any one question.

Disen Liao and colleagues at the University of Waterloo and Vector Institute formalize this as compositional safety risk. Their illustrative case involves a hidden objective of selling counterfeit concert tickets, with separate queries about ticket security features, online marketplaces, listing photography, and high-quality home printing. Each topic can be used for harmless work on its own. The risk arises when an external composer assembles the answers toward the concealed objective.

Magnet treats this process as capability accumulation. A capability is an artifact produced at one step of an objective, evidenced by model responses and tool-call results, that can combine with artifacts obtained elsewhere. In agentic pipelines, saved files and database records offer another path because later tasks can read them without the conversation history that came before. An injected instruction planted in one session can therefore activate in another.

Anthropic’s September 2026 threat intelligence report describes a real case with actors attributed to GTG-87001. They split work across multiple sessions so that no single session revealed their full intent. Many requests were blocked, but the cross-session trajectory succeeded where individual sessions were stopped. A blocked request was not the same as a stopped operation.

Why larger models make each fragment of a decomposed attack more dangerous

Better models can make this attack class worse. The same improvements that help a model give a useful answer can make an innocuous-looking fragment more complete, precise, and easy to combine with other fragments. An attacker does not necessarily need a model to accept an explicitly harmful request. Better answers to allowed subqueries can increase the yield of the attack.

Liao and colleagues establish a conditional risk-transfer bound. Their result applies when a reference environment already contains dispersed evidence that supports a risky reconstruction. Under that condition, the model’s excess loss on allowed subqueries controls the gap between deployed and reference composed risk. The condition is central to the finding. It concerns evidence already available in pieces and the model’s ability to connect it.

The same study finds that wider transformers assign lower loss to held-out instructions recoverable from supporting facts, even when those instructions never appear verbatim in training. Its evaluation of Qwen3 and Gemma3 family members also finds that larger models can produce greater harmful-capability uplift under a fixed decomposition-composition pipeline. A product upgrade can thus make the fragments an attacker has collected more useful without improving a detector that still judges each session on its own.

What existing benchmarks and frameworks were built to measure — and what they miss

Existing prompt injection benchmarks mostly test attacks in which the injection and its effects on behavior take place within one session. XSPI-Bench addresses that limitation through a taxonomy-guided benchmark built on a stateful agent system with persistent memory. Its cross-session cases test a lifecycle that single-session evaluations do not capture, rather than merely adding harder prompts to the same setting.

Crew Scaler’s framework analysis finds a similar gap in what governance covers. It scores sixteen security and governance frameworks against 193 distinct multi-agent system threat items across nine categories, and none of the frameworks covers more than half the items in any category. Non-Determinism and Data Leakage are the most under-addressed domains. Those omissions are especially relevant when a harmful result depends on context accumulated across requests that may not be reliably repeatable.

The analysis identifies the OWASP Agentic Security Initiative as the leader in overall coverage and the CDAO Generative AI Responsible AI Toolkit as the leader in development and operational coverage. Even the leading frameworks leave the cross-session surface structurally uncovered. Strong results against a different threat model do not show that a deployment can detect incremental extraction across sessions.

Three detection architectures competing to cover the cross-session surface

The emerging approaches differ over where the signal lives. Intent-aligned retrieval looks for a hidden task that links separate requests, capability tracking checks what the interactions produced, and activation-level monitoring looks for changes in what the model understood. These are distinct architectural bets. They can also form complementary layers, so the available evidence does not justify declaring a single winner.

IntentAlign-MiniLM retrieves cross-session intent neighbors from a query bank before classification. Standard semantic retrieval may find questions that sound alike but do not help with the same task, while intent-aligned retrieval looks for requests that seem different but help with a shared task. Liao and colleagues report that their 22M-parameter retriever outperforms much larger embedding models on held-out intent retrieval. It also achieves the best learned-retriever harmful recall across the tested guardrails. The comparison applies only to those tests.

Magnet instead grounds detection in accumulated capabilities. It aggregates evidence at the user-ID level and examines model responses and tool-call results that can combine toward a harmful whole. This changes the central question. Rather than asking only what a user appears to want, the detector asks what the interactions have made possible. That distinction is especially useful when the prompt framing hides the purpose but the produced artifact remains concrete.

A research paper describes activation-level drift monitoring as a layer that can work alongside the others. DeepContext detects semantic drift in what was said, whereas activation-level approaches seek drift in what the model understood, including cases where surface text remains benign. This adds a different view of the interaction rather than resolving the choice between intent and capability. To work across sessions, all three approaches need persistent data structures that link evidence from one interaction to another. Most deployments do not currently operate that infrastructure.

The hardest objections to cross-session monitoring and what they require defenders to accept

Liao and colleagues identify false positives as the principal calibration challenge for a query-bank defense, because unrelated past requests can be retrieved together and made to resemble a harmful objective. Harmless research can create neighborhoods that look suspicious too. Correlation is therefore an early signal that requires distinguishing a coherent attack from an accidental grouping, even with better retrieval.

Magnet shows that single-dimension compression loses discriminative power when benign sessions dilute the attack signal or an adversary floods the history with strategic duplicates. A rare suspicious artifact can disappear into the average. A deployment’s representation of past interactions must preserve the evidence needed to distinguish harmful accumulation from ordinary use.

Why not let a cross-session judge see every interaction? That answer requires persistent, user-linked query stores that most deployments do not operate, and those stores create privacy risks of their own. Context-fracture attacks make the requirement harder by spreading intent-bearing pieces across sessions and time so they never share a single window. Extending the window is not cost-free.

Shared agent memory further complicates attribution because poisoned context can affect multiple users through multiple agents. Detection must then connect agents, sessions, and users over days or weeks. A user-only view can miss the shared dependency. Monitoring must identify suspicious requests and trace how context persists and influences later work.

There is also an unresolved methodological choice between intent and capability. Magnet argues that intent, grounded in prompt framing, creates a chokepoint an adversary needs to bypass only once, while concrete outputs and tool-call effects are harder to obscure. Intent-aligned retrieval makes a different bet about reconstructing the hidden task. Neither position makes the trade-offs disappear. Adoption requires persistent evidence, privacy controls, and careful calibration.

How can organizations detect extraction attempts across vendor AI deployments today?

For third-party risk management, information security, privacy, and Legal teams, the operational response begins with persistent, entity-linked interaction records. Magnet’s user-ID-level aggregation requires session outputs and tool-call results to remain attributable after a conversation ends. Without those records, reconstruction has no evidence. The requirement is broader than retaining request text, because an artifact produced through a tool can be central to the attack even when it never appears in a user-visible query.

IntentAlign-MiniLM’s findings support retrieving requests by the hidden task they serve. Capability tracking adds the responses, artifacts, and tool-call effects to that evidence bundle. The layers answer different questions about the same trajectory. Together, they provide criteria for evaluating monitoring tools without assuming that a general-purpose embedding model or prompt-local filter covers the whole problem.

For system prompt confidentiality and data exfiltration risk detection, that distinction sets an important limit on the claim. Cross-session correlation can reveal suspicious accumulation, but the cited findings do not establish that it guarantees confidential instructions will remain secret. Evidence of an attempted extraction is not proof of a completed leak. Tool evaluation therefore needs to distinguish the ability to connect suspicious requests from the ability to track what the deployment actually disclosed or produced.

When choosing a framework, teams also need an explicit gap analysis to check what it misses on this attack surface. Crew Scaler’s findings show why a chosen governance framework cannot stand in for evidence of cross-session coverage. Continuous monitoring must cover the time before and after vendor AI deployment and carry on beyond an assessment. Traditional security tools and bolted-on compliance checklists do not have the mechanisms described here to compare intent neighborhoods and track capability accumulation.

The final boundary is the vendor ecosystem itself. Anthropic’s cases with GTG-87001 and GTG-84002 show cross-session accumulation in vendor-hosted systems, where enterprises may be unable to access session logs directly. Monitoring must cover self-hosted models, internal applications, and the whole vendor AI asset inventory, including plugins, agents, and connectors that come through third-party relationships. Dedicated, continuous AI risk intelligence must cover that ecosystem because an internal session boundary is not the boundary of an attacker’s operation.