Automated Dual-Framework Scoring of Third-Party AI Vendors Against OWASP LLM Top 10 and NIST AI RMF

Vendor AI risk now moves faster than the tools most TPRM programs use to assess it. The answer is automated dual-framework scoring: map every vendor AI component to the OWASP LLM Top 10 for specific technical risks, weight those findings through NIST AI RMF's governance functions, and monitor continuously. The two frameworks occupy different layers. Neither closes the vendor risk loop alone.

Why traditional TPRM fails when vendors embed AI

AI capabilities are embedded throughout the vendor ecosystem, in customer service chatbots, fraud detection engines, code generation tools, and healthcare diagnostics. That integration happened faster than governance tools were designed to evaluate it. Traditional security questionnaires were built for a different era, when a vendor's risk profile was stable between review cycles and the relevant controls were things like encryption standards and access policies.

The structural blind spot is specific. They don't surface whether a model discloses sensitive information under adversarial prompting. The questions that would reveal these risks simply don't exist in the standard assessment forms.

Shadow AI sharpens the problem. SaaS vendors now add AI features to products that have already cleared procurement review, often without announcing the change to customers. The buying organization's risk exposure grows while its documentation stays frozen. Enterprise AI risk intelligence platforms like Promptarmor address this gap by watching vendor AI changes and scoring them against AI-specific risk frameworks, so you see when a vendor adds or changes AI capabilities after deployment.

The velocity problem compounds the structural one. New data flows get established. The buying organization learns about none of it until something breaks or a breach report lands.

What OWASP LLM Top 10 and NIST AI RMF each cover, and where each one stops

OWASP names the specific technical risks that a vendor AI system can expose. NIST AI RMF gives you the governance process for identifying, measuring, and managing those risks. One framework tells you what to watch for. The other tells you how to act on what you find.

The OWASP LLM Top 10 2025, released November 18, 2024, is a community-driven consensus list of the ten most critical security risks in applications built on large language models. It covers everything from prompt injection to unbounded consumption. It is not a theoretical taxonomy. It is a prioritized map of how production LLM systems are actually attacked.

NIST AI RMF operates at a different layer. It provides a process model, built on the functions GOVERN, MAP, MEASURE, and MANAGE, for treating AI risk across an organization, including third-party AI. But it does not itself name LLM-specific attack classes. That is a deliberate design choice, and it matters for vendor risk. Three NIST AI RMF subcategories are explicitly scoped to vendor relationships. GOVERN 6 covers policies and procedures for AI risks from third-party software, data, and other supply chain issues. MAP 4 covers risk mapping for all AI system components, including third-party. MANAGE 3 covers managing risks and benefits from third-party entities.

Neither framework alone closes the vendor risk loop. NIST without specific technical risk definitions produces governance process without a precise object to govern. A purpose-built vendor AI risk platform can score against both simultaneously, translating OWASP's technical vulnerabilities into NIST's governance structure so TPRM, InfoSec, and Legal teams move from identifying a risk to assigning accountability and remediation within a single composite score.

The OWASP LLM Top 10 risks that matter most in vendor relationships

Several OWASP LLM 2025 entries carry disproportionate weight in vendor assessments because the buying organization has no direct control over the vendor's model, training data, or agent permissions. External scoring is the only lever available. Prompt Injection, LLM01, holds the top spot for the second consecutive edition. LLMs cannot separate instructions from data in the same channel. An attacker who can reach the model through any input path, including content the model retrieves, not just what a user types, can redirect its behavior. The 2025 edition highlights multimodal injection as an additional vector.

Sensitive Information Disclosure, LLM02, jumped from sixth to second place. Models can memorize training data and reproduce fragments of it, including PII, credentials, and proprietary content. In a vendor context, the buying organization's data may sit in that training set or context window, and the organization has no visibility into that exposure without external scoring.

Supply Chain, LLM03, rose from fifth to third. A compromised component anywhere in that chain affects every downstream application built on it. If you pull in third-party models, you inherit problems in the training data and model weights, often without realizing it. Data and Model Poisoning, LLM04, expanded from the narrower 2023 "Training Data Poisoning" to cover model poisoning across pre-training, fine-tuning, and embedding data. A vendor that fine-tunes on a tampered dataset delivers backdoored behavior to every customer using that model version.

The remaining five entries have vendor-relationship dimensions but carry lower scoring weight in most TPRM contexts. System Prompt Leakage (LLM07), Vector and Embedding Weaknesses (LLM08), Misinformation and Hallucinations (LLM09), Improper Output Handling (LLM05), and Unbounded Consumption (LLM10) round out the ten. Platforms that track vendor AI changes and score against these entries can flag shifts in a vendor's supply chain, model permissions, or data exposure, catching risks like newly integrated plugins or expanded agent capabilities before they breach compliance or data boundaries.

How the two frameworks cross-map into a composite vendor score

The cross-mapping between OWASP entries and NIST AI RMF functions is now a first-party artifact, not a practitioner inference. That makes automation built directly on those mappings structurally defensible. You can audit the logic.

Concrete mapping examples come straight from the OWASP LLM Top 10 2025. LLM03 Supply Chain maps to NIST AI RMF GOVERN 6, third-party risk, and MANAGE 3. These mappings turn abstract alignment into something a scoring engine can execute.

A scoring model translates each OWASP entry into assessable vendor controls, then weights those controls by NIST AI RMF function. The result is a composite score across domains including security posture, compliance, data handling, model provenance, and operational resilience. Critical-tier controls that score at the floor serve as automatic disqualifiers. A vendor with no enumerated, verified position on training data use is an automatic disqualifier for any engagement touching customer or employee personal data.

And one thing deserves emphasis before moving on. An AI Bill of Materials, or AIBOM, is the technical baseline that makes composite scoring auditable rather than impressionistic. It inventories every model, dataset, adapter, library, and plugin in the vendor's AI stack with verified provenance, signatures, and pinned versions, giving the scoring engine a concrete object to assess rather than self-reported documentation.

What happens when vendor AI risk goes unscored: real incidents

The incidents and legal outcomes that follow show that deploying a third-party AI tool transfers neither the risk nor the liability to the vendor. The buying organization is accountable for outcomes it could have assessed. EchoLeak, tracked as CVE-2025-32711 with a CVSS score of 9.3, emerged in June 2025. It was a zero-click indirect prompt injection in Microsoft 365 Copilot that exfiltrated data from a crafted email with no user action required. A textbook combination of LLM01 and LLM02 in a widely deployed vendor tool.

ForcedLeak, with a CVSS score of 9.4, followed in 2025. An indirect prompt injection in Salesforce Agentforce, delivered through a Web-to-Lead field, led to CRM data exfiltration. It combined LLM01 with LLM06, excessive agency. The case illustrates why LLM06 scoring must cover third-party agentic tools, not only self-hosted models.

In March 2026, attackers published two malicious versions of LiteLLM, an open-source package developers use to connect applications with different AI model providers. They used stolen maintainer credentials to publish to the official PyPI account, so the malicious versions looked like legitimate routine updates. This was a direct instance of LLM03 executed against a widely used AI infrastructure component.

Then came the Home365 settlement with the Pennsylvania Attorney General in May 2025. A property management company that deployed a proprietary AI platform faced enforcement action after the platform contributed to maintenance delays and unsafe housing conditions. The settlement established that deploying organizations bear accountability for third-party AI outcomes, a legal precedent that elevates vendor AI scoring from best practice to organizational necessity.

The objection that automated scoring produces compliant-looking paperwork rather than actual risk reduction

The objection is real and must be answered rather than dismissed. Automated scoring systems can produce risk ratings that satisfy a procurement checkbox but miss the contextual factors that decide whether a vendor's AI posture is actually adequate. Both OWASP LLM Top 10 and NIST AI RMF are voluntary guidance frameworks. They carry no regulatory enforcement mechanism and no certification. Scoring vendors against two non-certifiable frameworks may generate documentation that reads as due diligence without creating legal protection equivalent to a certified standard. That is a fair criticism.

The black-box critique is also structurally valid. If compliance teams cannot explain how a specific risk rating was calculated, they cannot defend it during an audit or justify a vendor decision under legal scrutiny. This transparency problem affects automated platforms generally, not just those in the vendor AI space. But the counter-argument is not that automation is sufficient. Automated dual-framework scoring holds up when it is transparent in its weighting logic, continuous rather than point-in-time, and treats the composite score as a trigger for human review at defined thresholds, not as a replacement for judgment on high-consequence decisions.

Why point-in-time scoring is insufficient and continuous monitoring is the operational requirement

A vendor's dual-framework score at onboarding is a snapshot of a moving target. The score must be sustained by continuous monitoring because vendor AI risk profiles change between review cycles in ways that questionnaires cannot detect. Shadow AI is the most common mechanism of undetected drift. The buying organization's risk exposure can increase substantially with no visible change in vendor status. This is the exact failure mode a static score cannot catch.

The tools for catching it exist. NLP tools can scan vendor documentation, terms of service, and sub-processor lists to detect undisclosed AI usage. A point-in-time score would have missed it completely.

NIST AI RMF's expanded guidance through 2025 made the framework's position explicit. Third-party risk is treated as a primary and ongoing concern, not a procurement-phase checklist item. TPRM programs that assess only at onboarding are operating with a gap that regulators and auditors are increasingly prepared to identify.

How to implement dual-framework scoring in a TPRM program: the operational steps

Implementing dual-framework scoring requires a defined sequence. Establish an AI asset inventory of vendor components, map each component to OWASP entries, weight controls by NIST AI RMF function, set hard gates for critical-tier findings, and build continuous monitoring into the vendor lifecycle rather than treating assessment as a procurement-phase event. Step 1 is the AI asset inventory. Before scoring, establish what is being scored. Without this baseline, the scoring engine operates on self-reported information the vendor controls.

Step 2 is OWASP control translation. Map each of the ten OWASP LLM 2025 entries to specific, assessable vendor controls. Prioritize LLM01, LLM02, LLM03, LLM04, and LLM06 for vendors running agentic tools or handling personal data.

Step 3 is NIST AI RMF weighting. Assign governance weights to each control domain using the GOVERN, MAP, MEASURE, MANAGE function structure. The vendor-relationship subcategories, GOVERN 6, MAP 4, and MANAGE 3, receive elevated weight in a TPRM context.

Step 4 is hard gates. Define automatic disqualifiers independent of composite score. Any vendor without a verified, documented position on training data use is a disqualifier for engagements touching customer or employee personal data.

Step 5 is tiering and continuous monitoring. Tier vendors by composite score and AI exposure. Automate alerts for new model integrations, sub-processor additions, and AI-related security incidents.

Workday's use of the AI RMF as a common reference point for cross-functional teams and for building a questionnaire to assess third-party AI tools it procures shows what makes this work. Cross-functional alignment, across product, legal, and responsible AI teams, is a prerequisite for the governance weighting to carry organizational authority. The tooling matters less than the commitment behind it.

Platforms that automate dual-framework scoring against OWASP LLM Top 10 and NIST AI RMF

A small category of platforms now automates dual-framework scoring against both OWASP LLM Top 10 and NIST AI RMF, combining vendor AI asset discovery with continuous risk monitoring. That combination is what the preceding sections establish as necessary. Promptarmor is built specifically for TPRM, InfoSec, Privacy, and Legal teams managing third-party AI risk. It assesses and continuously monitors AI assets across the entire vendor ecosystem, surfacing emerging threats before they breach compliance or business continuity. For TPRM teams, that means the discovery and the scoring live in the same workflow.

Censinet takes a narrower path, focused on healthcare vendor risk. The company partnered with Renown Health in February 2025 to automate IEEE/UL 2933 compliance checks for new AI vendors, aligning with the TIPPSS framework covering Trust, Identity, Privacy, Protection, Safety, and Security. Teams outside healthcare will find the framework coverage limited.

What dual-framework scoring cannot do, and what fills the remaining gap

Dual-framework scoring closes the visibility gap that traditional TPRM leaves open. It does not transfer accountability or substitute for the judgment calls that remain inherently human. The gap between a high composite score and actual safety is widest for vendors deploying agentic AI with real-world tool access.

The Home365 outcome establishes the accountability ceiling. Scoring gives you evidence. It does not absolve you.

The remaining judgment gap is real. Composite scores can flag that a vendor's LLM06 posture is weak. No algorithm can substitute for that decision. The regulatory direction narrows the voluntary-framework objection over time. The framework's voluntary status is the current position, not necessarily the durable one.

The complete program combines four elements. None of these four elements is optional. If the goal is a defensible vendor AI risk posture rather than compliant-looking documentation, the program needs all of them.