AI Security Due Diligence in Vendor Contracts

A vendor contract built for a CRM or a payroll system assumes the software behaves the same way every day. AI vendors break that assumption: the model changes, the training data pipeline may pull in your inputs, and the outputs shape real decisions with real consequences. This piece walks through what due diligence has to look like now, from the questions worth asking before signature to the clauses that actually hold vendors accountable once the ink dries.

What makes AI vendor risk structurally different from ordinary third-party risk

Start with the basic mismatch. Standard third-party risk management assumes a system that stays put. You test it once, you check a few boxes, and you trust that the vendor's behavior in month twelve looks like its behavior in month one. AI vendors don't offer that guarantee, and four specific properties explain why.

First, there's the training feedback loop. Customer inputs can become training signal. That means proprietary data fed into a vendor's tool might shape a model that other customers, maybe even competitors, end up using. Second, model outputs are probabilistic, not fixed. The same prompt can produce different answers on different days, and even the vendor often can't fully explain why a given output came out the way it did. This isn't a bug to patch; it's how these systems work.

Third: subprocessor chains run deep. The vendor you signed with might sit on top of a foundation model from one company, hosted on cloud infrastructure from another, fine-tuned by a fourth. Your contract reaches the first party. It rarely reaches the rest.

Fourth, and maybe the thorniest: output liability. If an AI tool wrongly denies someone credit, screens out a qualified job candidate, or flags the wrong medical concern, who's on the hook? The vendor who built the model, or the company that deployed it? Most agreements are silent on this question, so the answer only emerges after something has already gone wrong.

Supply chain risk compounds all four. OWASP's LLM Top 10 now ranks supply chain vulnerabilities among the top three risks facing production AI systems, a jump that reflects how fast this threat has grown. Poisoned training data, compromised pre-trained models, vulnerable third-party packages: these sit upstream of the relationship you can actually see and audit. Agentic AI raises the stakes further. Systems that pull in plugins and third-party "skills" at runtime create dependencies that shift faster than any static review can track.

Then there's shadow AI. Plenty of platforms default to using customer inputs for model improvement unless someone actively opts out, a term usually buried in a service agreement rather than raised in the sales pitch. IBM's 2025 Cost of a Data Breach Report found that organizations tolerating unapproved AI tool use face meaningfully higher breach costs when incidents hit. Put it together and a diligence process built around certifications and standard data clauses misses most of this. The gaps aren't small oversights. They're structural.

The regulatory obligations that now land on the deployer, not just the vendor

Here's a common assumption worth challenging: that the company building the AI system carries most of the legal weight. That's not how it works, at least not under the EU AI Act.

The Act splits responsibility on purpose. Providers have to build high-risk systems to a legal standard. Deployers, meaning the enterprises actually buying and running these systems, carry separate duties under Article 26: human oversight, ongoing monitoring, and log-keeping among them. Here's the catch. If the vendor hasn't handed over the legally required instructions for use, the deployer literally cannot comply with Article 26(1). That turns vendor documentation into a hard prerequisite before go-live, not a nice-to-have you chase down later. Penalties for non-compliance with high-risk AI obligations run up to €15 million or 3% of global annual turnover, a serious tier even though it's not the Act's absolute maximum. The core compliance deadline for Annex III high-risk systems got pushed to December 2027 under the EU AI Omnibus agreement approved in June 2026, but that extension shifts when enforcement starts, not what Article 26 requires.

NIST's AI Risk Management Framework tells a similar story on the U.S. side, even though it's technically voluntary. Released in January 2023 and expanded with the Generative AI Profile (NIST AI 600-1) in July 2024 to deal specifically with vendor-supplied models and large language systems, the framework shows up in enforcement guidance from the FTC, SEC, and CFPB. Voluntary doesn't mean consequence-free. Federal contractors face direct alignment requirements, and prime contractors are liable for the AI compliance of their subcontractors and commercial model vendors, no exceptions carved out for "we didn't know." A preliminary draft released in December 2025, NIST IR 8596, bridges the AI RMF with the Cybersecurity Framework 2.0, a signal that AI risk and cyber risk are getting treated as one and the same discipline going forward.

Then there's ISO/IEC 42001, the first international standard for AI management systems, covering risk assessment, data governance, and monitoring. Worth being precise about what certification actually means here: it doesn't validate that a model is safe. It validates that the vendor runs a systematic process for managing AI risk. That's a narrower claim than it sounds, but it's still useful. Amazon (for Bedrock, Amazon Q Business, Amazon Textract, and Amazon Transcribe specifically), Anthropic, Snowflake, and ServiceNow have all achieved certification, and the list keeps growing, especially as it becomes a baseline expectation in financial services and healthcare procurement. For a procurement team, a certificate is a signal you can verify without auditing the vendor's internal processes yourself.

Federal contractors in the U.S. face their own layer: disclosing every AI system in use, aligning documentation with the NIST AI RMF, and reporting security incidents to CISA and contracting officers within 72 hours. Those obligations flow down to subcontractors and AI model vendors too. The thread running through all of this: regulatory duty that lands on the deployer only means something if the contract backs it up. If a vendor fails to deliver the documentation, access, or disclosure a deployer needs, the deployer is still the one standing in front of the regulator.

What pre-signature assessment should actually examine

Certifications set a floor. They don't set a ceiling, and treating them as the finish line is where a lot of diligence processes quietly fail.

Most enterprise buyers already require SOC 2 before signing, and that's a reasonable baseline. But SOC 2 was built for infrastructure and data handling, not for model behavior, training data provenance, or bias mitigation. It answers "is the vendor's environment secure," not "does this model do what it claims to do, consistently, without hidden risk." Worth checking, too, whether the report is Type I or Type II. Type I only reviews whether controls are designed properly; Type II checks whether they actually operate that way over time. For any AI tool carrying medium or high risk, a Type I report alone should be a dealbreaker. A vendor with its house in order can usually produce a SOC 2 Type II report within two weeks of a request. If they can't, that delay tells you something. Where a vendor holds ISO 42001 certification, check the scope directly: which systems and processes did the audit actually cover, and which ones did it leave out?

Penetration testing needs a similar level of scrutiny. Ask for a test on the AI system's infrastructure completed within the last twelve months; AI infrastructure moves fast enough that anything older isn't a reliable signal. A vendor who won't share at least an executive summary is either hiding something material or never ran a test worth mentioning. Standard pen tests, the kind built for network and application layers, often skip the AI-specific stuff entirely. Ask directly whether testing covered prompt injection, adversarial input handling, and output manipulation. If it didn't, the test tells you less than it appears to.

Subprocessor mapping matters just as much. Every AI subprocessor needs a name: the foundation model provider, fine-tuning services, cloud infrastructure, data annotation vendors. Ask whether any part of the system trained on publicly scraped data of uncertain origin, a question that touches both data poisoning risk and intellectual property exposure. Confirm, too, whether the vendor tracks and responds to vulnerabilities showing up in third-party model components and packages, or whether that's someone else's problem in their view.

Training data practices deserve a direct question, asked in writing: does the vendor use customer inputs or outputs to train or improve its models, and is that opt-out, opt-in, or simply non-negotiable? Sales conversations aren't binding. Only the written policy is, so get it in writing before signature, not after. It's also worth understanding how customer data gets isolated from other tenants, especially for any feature involving fine-tuning or personalization.

Last piece: regulatory classification. For anything touching EU markets, ask the vendor to state in writing whether its system counts as high-risk under the AI Act and whether a conformity assessment has already happened. For U.S. federal contractors, confirm disclosure and documentation requirements are met before onboarding starts. None of this is optional groundwork. Ongoing monitoring, the kind covered later, can't function without this baseline documentation gathered here, before terms are agreed.

The contract clauses that address what certifications alone cannot

Certifications tell you about process. Contracts tell you what happens when something breaks, and that's where most AI vendor agreements still fall short.

Data ownership needs to be split into three distinct layers, each addressed on its own. Input data: who owns and controls what gets fed into the system. Output data: who owns what the model generates, including insights and recommendations. Derived data: whether the vendor can use any of the above to retrain its models or build other products entirely. A no-training clause should survive termination of the contract, because retraining that already happened using your data doesn't reverse itself just because the relationship ended. Push for a requirement that the vendor notify immediately if customer data turns up in outputs delivered to other customers, a real failure mode in multi-tenant systems, not a hypothetical one. Data retention limits and secure destruction (or verified de-identification) at contract end round out this section.

Model change notice is another gap most standard SaaS agreements never anticipated. Require advance written notice before a vendor retires, replaces, or materially changes the underlying model. Define what counts as a material change and spell out what evaluation rights the customer keeps when one happens. A model retrained six months after signature isn't the system that got assessed, so the contract needs to treat that as an event, not a footnote.

Liability caps built for service outages don't map cleanly onto AI-generated harm. Negotiate carve-outs specifically for data breach and misuse, IP infringement stemming from AI-generated outputs (a real risk given how many models train on data of murky origin), gross negligence and willful misconduct, confidentiality breaches, and regulatory fines triggered by a vendor's failure to deliver required documentation. For data-related liabilities specifically, some deployers are pushing for a "super-cap," a higher ceiling, often set as a multiple of annual fees, as an emerging negotiating position for high-risk AI use cases.

Incident response timelines need teeth too. "Notify promptly" doesn't hold up against the SEC's cybersecurity disclosure rule, which runs on a four-business-day clock for material incidents. Vendor contracts need to match that pace, not lag behind it. Spell out exactly what triggers notification, who receives it, the timeframe, and what forensic evidence has to be preserved and for how long. The vendor should also be contractually required to support the customer's own regulatory disclosure process, not just manage its own separately.

Audit rights only matter if they're usable in practice. Specify frequency, scope, notice period, and what happens if an audit turns up gaps that need fixing. Require the vendor to maintain any stated certifications, ISO 42001, ISO 27001, SOC 2, for the full term of the contract, and to notify if a certification lapses or its scope narrows.

High-risk classification deserves its own clause. Require the vendor to disclose if its system qualifies as high-risk under applicable law, and to update that disclosure if the classification shifts due to product changes or new regulation. Retain the right to walk away without penalty if a system that wasn't high-risk becomes one and the vendor can't show it's ready for that shift.

Subprocessor controls close the loop. Require prior written consent before the vendor adds or swaps out material AI subprocessors, mirroring the logic GDPR already applies to data processors but extending it to model-level dependencies. Extend the same security and data use obligations, contractually, to every named subprocessor down the chain.

Why contract-stage diligence must connect to ongoing monitoring

Here's the trap: treating AI vendor diligence as a single event tied to procurement, something that happens once and gets filed away. AI systems don't sit still. Models get retrained, updated, deprecated, replaced. Subprocessors change. New agentic capabilities get bolted onto products that were approved under an earlier, simpler version of themselves.

That raises an obvious question. A vendor that passed every check at signing, what happens when the underlying system looks nothing like it did six months later, and no formal change-of-service notice ever gets triggered? Forrester's 2026 threat intelligence outlook names AI agent risk as a defining concern for security leaders precisely because agentic systems create dependencies that sprawl and shift faster than any point-in-time review can keep up with.

Monitoring after signature has to pick up where the contract-stage work leaves off: tracking subprocessor changes, watching for model updates that alter behavior, and confirming that certifications named in the contract are still valid and still cover the right scope. None of that works without the baseline established before signature. The pre-signature assessment isn't a gate you pass through once. It's the reference point everything after it gets measured against.