LLM Security Review

AI-Specific Gaps in Traditional TPRM Programs

AI systems violate the three core assumptions traditional vendor risk assessment depends on.

· 10 min read
Cover illustration for “AI-Specific Gaps in Traditional TPRM Programs”
AI Security Governance · October 9, 2026 · 10 min read · 2,285 words

Traditional third-party risk management rests on three assumptions, and AI vendors break all three by design. The first is that vendors are static: once approved, a vendor's risk profile holds steady until the next scheduled review. The second is that software logic is deterministic: code can be read, architectures can be audited, and a trace exists from input to output. The third is that compliance outputs are auditable: a document produced during assessment accurately describes what the system does, today and for the foreseeable future.

AI vendors violate each one. A foundation model can be swapped, a subprocessor can be added, and data-handling terms can shift, all without triggering a single contractual notification. The reasoning behind an AI model's output often cannot be fully explained even by the company that built it, which creates an accountability gap no amount of auditing experience can close. And because AI outputs are probabilistic, a model that passed every test at onboarding can produce unreliable results months later with no code change. This is not a maturity problem that better staffing or faster questionnaires will fix. The framework itself assumes a kind of stability and transparency that AI systems don't offer, and every gap described in this piece traces back to that mismatch.

Point-in-time assessments cannot track vendors whose risk profile changes between reviews

An assessment completed at onboarding captures a single moment. For traditional software, that moment tends to hold, because the vendor's architecture doesn't change on its own. For an AI vendor, that moment starts decaying the day it's recorded.

Model drift is the clearest version of this. An AI vendor's underlying model ingests new data and behaves differently over time, with no deployment event, no version bump, nothing for a change-management process to catch. Layered on top of that are the subprocessor and data-handling shifts that happen quietly between review cycles: a new foundation model swapped in, a new API integration added, retention terms revised, none of it requiring notice to the customer under standard SaaS agreements.

Timing makes the problem worse. ProcessUnity's State of Third-Party Risk Assessments Report found that a large share of organizations take an average of four months to complete a single vendor risk assessment. An AI vendor can change materially within that window, before the assessment reviewing it has even closed. And there's rarely a contractual mechanism to catch it: "material AI change" is almost never defined in vendor agreements, and almost never tied to an advance notification requirement. The missing clause means there is no defined trigger for a re-assessment when the model underneath the vendor changes. The review cycle keeps running on schedule. The thing it's supposed to track doesn't wait for it.

Standard vendor questionnaires were written for SaaS that stores files, not software that executes autonomously

Even if every vendor review ran continuously instead of annually, a deeper mismatch would remain. Standard vendor questionnaires were built to capture data residency, access controls, breach notification service levels, and certifications, a set of questions designed for systems that hold data passively. An AI vendor doesn't just hold data. It acts on it.

A few gaps matter enough to name directly. Training data provenance is one: does data sent to the vendor get used to train models that other customers, including competitors, might later query? Training opt-out is another: without an explicit contractual carve-out, every prompt and document sent to an AI vendor becomes potential training material by default. Prompt injection defenses matter too: red-teaming against prompt injection, the mechanism attackers use to hijack model behavior or pull sensitive data out of a third-party AI tool, rarely appears as a line item on a standard questionnaire. And runtime behavior sits at the center of all of it: what does an AI agent actually execute inside the customer's environment once it's authorized? That's a categorically different question than where a file is stored.

Model bias belongs on this list as well, particularly for regulated use cases. Under the EU AI Act, a deployer using a third-party hiring or credit-scoring algorithm carries its own obligations, including human oversight, monitoring, and fundamental rights impact assessments under Article 26, with penalties attached for falling short. Most of the heavier obligations, including bias mitigation and conformity assessment, sit with the provider rather than the deployer, but deployer liability still applies, and standard questionnaires contain no bias-assessment items.

Certifications don't close this gap either. SOC 2 and ISO 27001 establish a solid baseline for information security, but neither one touches training data provenance, model transparency, algorithmic bias, or how a vendor manages model change. A vendor can hold both certifications in good standing while every AI-specific risk in its product goes unassessed. The questionnaire format itself is built for the wrong kind of system.

N-party AI dependencies expand the vendor footprint beyond what any questionnaire maps

Even a perfect AI-specific questionnaire, applied faithfully to every named vendor, leaves a structural blind spot: AI tools almost never operate alone. A SaaS vendor that passes a TPRM review may itself call an external foundation model API, route customer data through a specialized inference subprocessor, and depend on a separate embedding service for search or retrieval. That chain never appears on the questionnaire for the vendor the TPRM team actually assessed.

Exiger's analysis of sub-tier opacity in supply chains gives a sense of how deep this kind of blind spot can run. An automotive supplier traced a critical semiconductor path through three intermediaries and found that one was majority-owned by an entity on the EU sanctions watchlist. Its own Tier 1 partner had no idea. AI subprocessor chains carry the same structural invisibility: the risk sits several hops away from the vendor whose name is on the contract.

Regulation has tried to address this through contract law. GDPR requires that processors pass the same data protection obligations down to every subprocessor, in writing, with the controller's prior authorization, so a chain of accountability gets built on paper. Many organizations haven't actually satisfied that requirement across their full AI tool portfolio, so the paper chain and the real chain don't match. An AI Bill of Materials, documenting the base models, external APIs, and data sources a given AI product actually depends on, is starting to emerge as a due-diligence expectation precisely because standard vendor scoping leaves this entire layer in the dark. One might argue that even a complete AIBOM only maps the vendors an organization knows to ask about.

Shadow AI means the vendor inventory TPRM programs assess covers only a fraction of the AI in use.

That question points at a bigger problem than assessment quality. A meaningful share of the AI tools running inside most organizations were never assessed at all, because they entered through a door TPRM doesn't watch.

The structural cause is organizational. Security teams own the questionnaire process. AI adoption, though, often runs through business units buying tools directly, sometimes on a company credit card, with data governance sitting somewhere in between the two functions without clearly owning either. A parallel AI vendor ecosystem now operates entirely outside the TPRM perimeter, in daily use yet unseen by the program meant to assess it.

There's a quieter version of the same bypass inside tools that were already approved. Once a SaaS application clears its initial review, the security assumptions behind that approval often sit unchanged for years. When the vendor later adds AI capabilities to that same application, those new capabilities get access to sensitive data and start performing automated actions inside a platform nobody is watching anymore, because it was cleared long ago. ProcessUnity has noted that due diligence backlogs already create dangerous blind spots even for vendors that are fully known and on record. Quietly embedded AI features add to that backlog, and the blind spot widens without anyone deciding it should.

Scanning OAuth permissions or watching network traffic can tell a security team that an AI tool exists somewhere in the environment. Neither approach can detect when an AI feature buried inside an already-approved application gets invoked. The Vercel breach in April 2026 showed what that gap can cost: a single AI productivity tool with excessive OAuth permissions became an attack vector into internal systems and identity infrastructure. And policy alone doesn't fix the underlying behavior. Most procurement professionals already use AI regularly in their work, but per Suplari's AI Readiness in Procurement benchmark, only 17% work at organizations with an enforced AI governance policy. A policy that exists on paper but isn't enforced doesn't shrink the shadow AI footprint. It just describes it.

The gaps described so far carry operational and security consequences. In 2026, they started carrying legal ones too, as regulatory frameworks introduced third-party AI due-diligence obligations specific enough to expose a program still running on generic questionnaires and periodic review cycles.

The EU AI Act sets the clearest stakes: penalties for non-compliance with provider obligations can reach into the tens of millions of euros, or a significant share of global annual revenue, whichever is larger. The US Treasury's Financial Services AI Risk Management Framework, introduced in February 2026, set out control objectives covering third-party AI vendor due diligence, independent validation of vendor AI models, and ongoing risk monitoring for AI models used in credit, fraud, and risk decisions. An annual questionnaire cycle cannot structurally satisfy monitoring obligations built around continuous validation. NIST's AI Risk Management Framework reinforces the same expectation in its GOVERN 6.1 function, which calls for policies that address AI risks tied to third-party entities, whether the AI was built in-house or bought from a vendor. NIST's framework stays voluntary. The EU AI Act and the Treasury framework don't, and together they push obligations well past where NIST's guidance stops.

Financial services, healthcare, and critical infrastructure carry the sharpest exposure here, given how directly AI touches credit decisions, patient data, and safety-critical operations in each. The regulatory floor has become specific enough that a TPRM program without continuous, AI-aware monitoring is now out of step with binding law, not just best practice.

Retrofitting Generic AI Tools Onto Legacy TPRM Workflows

A reasonable response to everything above is to point at AI-enhanced TPRM tools and say the industry is already solving this. More than half of organizations in the 2026 KPMG Global TPRM Survey are exploring exactly that kind of tool. Only 22% of them find the results very effective, and the rest report partial results at best.

The shortfall lies in where the AI models being used are pointed. ProcessUnity's analysis of task-level AI agents in TPRM names the failure mode directly: general-purpose AI tools produce outputs that are merely plausible. A plausible answer passes review, moves into a risk tier, shows up in a board report, and only gets caught as wrong when an auditor eventually pulls the thread. In a discipline where a wrong answer can mean missing a real exposure, a plausible-but-wrong answer is more dangerous than an honest, acknowledged gap would have been.

Deloitte's TPRM research points to the same friction from a different angle, naming cost, integration with legacy systems, and access to expertise as the leading barriers to real AI adoption in this space. When vendor documentation is incomplete or stale, which it routinely is for AI vendors that change faster than most, an AI risk-scoring model trained on that documentation produces unreliable scores that can steer decision-makers the wrong way. KPMG's survey backs this up from the data side: only a small minority of organizations report the highest tier of data quality in their TPRM programs, and high-quality data is the one precondition that lets AI-enhanced scoring actually beat manual review.

Putting a generalist AI tool on top of a questionnaire makes questionnaire analysis faster. That's a real improvement, as far as it goes. It does not add continuous monitoring for model drift, runtime behavioral detection, mapping of dependencies on other vendors further down the chain, or shadow AI discovery. Every gap named in the sections above stays open; it just gets processed a little quicker than before.

Closing the Gap: Continuous Monitoring, Purpose-Built AI Risk Assessment, and Contract-Level Controls

Closing these gaps calls for capabilities that simply didn't exist in legacy TPRM frameworks, because legacy frameworks were never built with AI vendors in mind. Three things stand out as load-bearing.

Continuous monitoring has to replace point-in-time assessment as the operating model, not just supplement it. Because AI vendor risk profiles shift between review cycles, assessment needs to run as an ongoing process: tracking model change notifications, subprocessor additions, data-handling policy updates, and observable behavioral drift as they happen, rather than waiting for the next date on the review calendar.

Risk assessment methodology built specifically for AI needs to replace the repurposed SaaS questionnaire. That means asking about training data provenance, opt-out terms, prompt injection red-teaming, runtime behavior, and bias controls as a matter of course, not as optional add-ons bolted onto an existing form.

And contract-level controls need to catch up to how AI vendors actually operate. That starts with defining "material AI change" in the agreement itself, tying it to an advance notification requirement, and extending subprocessor accountability through the full depth of the dependency chain rather than stopping at the first vendor named on the contract.

None of these three pieces work in isolation. Continuous monitoring without AI-specific assessment criteria just means watching for the wrong signals faster. AI-specific assessment without contract-level controls produces sharper findings with no mechanism to act on them. The structural mismatch described at the start of this piece, static assumptions applied to a vendor category that is never static, only closes when all three move together.

Sources

  1. The 2026 KPMG Global Third-Party Risk Management Survey
  2. Assessing AI’s impact on third party risk management

More in AI Security Governance