LLM Security Review

NIST AI RMF Implementation for Enterprise AI Systems

NIST's framework demands continuous governance cycles, not one-time compliance checklists.

Correspondent · · 13 min read
Cover illustration for “NIST AI RMF Implementation for Enterprise AI Systems”
AI Security Governance · September 16, 2026 · 13 min read · 2,988 words

NIST's own generative AI profile, AI 600-1, names prompt injection and supply chain integrity as distinct risk categories. The standard-setting body already knows generic controls don't cover these gaps, and named them separately on purpose.

What the four functions actually cover and why the architecture matters

The NIST AI Risk Management Framework, published January 26, 2023, splits into two parts. Part 1 lays out the conceptual groundwork, including trustworthiness characteristics that act as the evaluative lens for everything else, covering dimensions such as validity and reliability, safety, security and resilience, explainability, privacy, and fairness. Part 2 is the Core, the operational engine, organized around four functions: GOVERN, MAP, MEASURE, and MANAGE.

GOVERN is the odd one out. Where the other three functions apply to individual AI systems, GOVERN spans the whole organization. It covers culture, accountability, policy, and decision rights across the entire AI lifecycle, and it's built from six categories (GOVERN 1 through 6) breaking down into 19 subcategories. GOVERN 1.1 addresses the existence and implementation of AI risk policies as distinct requirements. Notice the wording there: existence and implementation are named as two separate things, not one. That distinction resurfaces later in this piece, and it's the hinge the whole argument turns on.

GOVERN 2 covers accountability structures across the organization's AI activities, including how responsibilities are assigned and upheld. Because GOVERN sits above the other three functions, it's what makes MAP, MEASURE, and MANAGE repeatable instead of ad hoc, one-off exercises that different teams handle differently depending on who's in the room.

MAP works at the level of a single AI system. It's where a company builds out its AI inventories, model cards, and system cards, documenting purpose, data sources, risk exposure, integration points, and how much a human is supposed to stay in the loop. MAP 5.1 asks organizations to document the likelihood and size of potential impacts, and not just what a model outputs on a given day, but second-order effects: people over-relying on a tool, skills eroding across a team, a business becoming quietly dependent on one vendor's API staying up.

MEASURE is where things get quantitative, or are supposed to. MEASURE 2.11 covers fairness and bias evaluation, which sounds straightforward until you realize it demands a data infrastructure capable of benchmarking consistently over time. NIST doesn't hand organizations a maturity model or a scoring rubric here. Companies have to build their own. That flexibility sounds generous. In practice, it's where most implementations quietly stall, and that stalling point deserves a full accounting later.

Risk treatment and incident response are handled under MANAGE. MANAGE 1.1 requires a treatment plan exist, and again, existence isn't execution. MANAGE 4.1 covers what happens after deployment: ongoing monitoring, appeal and override mechanisms for people affected by a bad output, decommissioning old models, and change management when something in the system shifts.

None of this runs once and stops. AI systems keep changing after they ship, through retraining, fine-tuning, or just drifting as the data they see in production diverges from what they were trained on. The framework's four functions are built to cycle continuously alongside that drift, not to close out like a checklist. NIST also publishes a companion Playbook with suggested actions tied to each subcategory. It's voluntary, updated roughly twice a year, and organizations are meant to pull from it selectively rather than treat it as gospel.

Voluntary is the word NIST uses. It's not quite the word that applies anymore for a lot of organizations.

For federal agencies and their contractors, alignment with AI RMF 1.0 is effectively required under Executive Order 14110 on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. Federal procurement contracts increasingly bake AI RMF alignment into vendor requirements, which pulls the framework past agency walls and into the entire contractor supply chain feeding those agencies.

Colorado gives a useful case study in how fast this can shift. The Colorado AI Act, signed in 2024, originally treated alignment with the NIST AI RMF or ISO/IEC 42001 as a piece of an affirmative defense (paired with a discover-and-cure process) and as grounds for a rebuttable presumption of reasonable care in enforcement actions brought by the state attorney general. Then came an amendment in May 2026, effective January 1, 2027, that repealed the original approach and replaced it with a narrower automated decision-making technology framework focused on transparency, disclosure, and a limited set of consumer rights. NIST AI RMF and ISO 42001 still do the heavy lifting under the amended law. They're the practical backbone for the documentation and disclosure duties the statute demands, even without the original liability shield attached.

The FTC, the CFPB, the FDA, the SEC, and the EEOC all reference NIST AI RMF principles in enforcement guidance, so the reach isn't confined to one sector. Organizations regulated under the EU AI Act often use the NIST framework as their internal operating model too, mapping its categories over to EU conformity assessment requirements, because building two parallel governance systems from scratch is a waste nobody wants to fund twice.

The financial sector got its own version on February 19, 2026, when the Treasury Department released its Financial Services AI Risk Management Framework, built directly on NIST's structure in partnership with the Cyber Risk Institute. It runs 230 control objectives across the AI lifecycle, with a dedicated section on third-party risk, a detail that comes back later.

Surveys on responsible AI adoption consistently find that most organizations remain behind on implementation, carrying regulatory exposure many of them haven't fully reckoned with yet. Meanwhile the market meant to help close that gap is expanding fast: global AI governance tooling was worth a modest sum in 2025, and forecasts put it at several times that figure by 2033, a 36% compound annual growth rate. Money is pouring into compliance infrastructure. Whether the underlying governance work keeps pace with the tooling is a separate question.

What genuine GOVERN implementation requires beyond writing policies

GOVERN is the prerequisite. Skip it, or half-do it, and MAP, MEASURE, and MANAGE get applied unevenly, deprioritized whenever budget season gets tight, or quietly abandoned the first time a deadline crunch hits.

Ownership comes first. Without a name attached to AI risk decisions, accountability spreads thin across a dozen teams and effectively belongs to no one. Primary ownership usually lands with General Counsel, the CISO, the Head of Risk, or a Chief Risk Officer, though plenty of other roles need to weigh in along the way. What matters is that the buck stops somewhere specific and traceable, not that everyone gets a vote.

An AI inventory has to exist before anything else works. It sounds basic, almost too basic to mention, but plenty of organizations find out mid-audit that three different departments have been building or buying AI tools independently, with no one at the top aware of the full picture. Without central visibility, risk management turns reactive by default: a problem appears in production, and only then does anyone go looking for what's actually running. A working inventory documents model purpose, data sources, risk exposure, integration points, deployment environments, and how much human oversight each system is supposed to get. Some organizations take this further and build an AI Bill of Materials, tracking provenance and dependencies across every model, dataset, and vendor relationship in play as a living map of the AI ecosystem itself.

Policy has to say something specific: where AI is allowed, which risk levels trigger an escalation, what counts as a deployment nobody's allowed to greenlight. GOVERN 1.1 asks that these policies exist. Operational GOVERN asks something harder: that they get enforced, audited on a schedule, and updated as the AI landscape shifts under everyone's feet.

Executive sponsorship isn't a nice-to-have here. NIST is explicit that governance efforts stall without visible commitment from leadership, on prioritizing risk, overseeing vendors, and funding the assurance tooling that makes any of this measurable. And AI risk shouldn't sit in its own silo, either. It belongs folded into the same enterprise risk conversation as IT risk, operational risk, and compliance, because a lot of the same people and the same incident response muscles apply. Policies also need updating for threats that are specific to AI and don't show up in a standard IT governance playbook, such as prompt injection and supply chain integrity risks that NIST's own generative AI profile explicitly names. A generic security policy written for traditional software simply doesn't have language for these.

How MAP and MEASURE work in practice, and where MEASURE breaks down

MAP takes the inventory GOVERN builds and uses it as raw material for system-level risk context. For each AI system in that inventory, someone documents purpose, data sources, who's affected, integration points, the deployment environment, and how oversight is designed to work day to day. MAP 5.1 asks for documentation of likelihood and magnitude of impact, including the second-order stuff: people leaning on a tool more than they should, skills atrophying, a downstream dependency nobody flagged until it broke. And because models keep learning and shifting after launch, MAP can't be a document that gets filed away once. It needs revisiting as the system itself changes.

MEASURE is supposed to turn what MAP identifies into something quantified, benchmarked, and auditable. MEASURE 2.11, covering fairness and bias evaluation, needs baseline data, a defined measurement method, and ongoing monitoring to mean anything at all. NIST doesn't prescribe a scoring approach, which gives organizations room to build something that fits their own risk profile. That same openness is exactly where the most common failure occurs.

An organization finishes GOVERN, finishes MAP, produces clean documentation for both, and then MEASURE quietly falls off the list. Not because anyone decided to skip it, but because the data infrastructure needed to benchmark risk consistently was never built. According to implementation research, 45% of organizations point to concerns about data accuracy or bias as a real hurdle, and 42% cite a lack of proprietary data to properly customize models. Both problems land directly on MEASURE's doorstep, since without clean data and a working pipeline, there's nothing to benchmark against.

MEASURE is live infrastructure made up of assessment documents alongside audit trails, model versioning, data lineage, and performance baselines that get checked against on a schedule, not once a year during an audit. It's live infrastructure: audit trails, model versioning, data lineage, performance baselines that get checked against on a schedule, not once a year during an audit. It also has to account for different trustworthiness characteristics separately, since output reliability, fairness across demographic groups, and resilience against adversarial inputs each need their own measurement approach. One evaluation method doesn't cover all three.

Qualitative measurement counts too. Expert review, red-teaming, structured testing, these are legitimate MEASURE outputs when a quantitative baseline doesn't exist yet. The framework accommodates both. The trouble is that a lot of organizations read the absence of quantitative infrastructure as license to skip MEASURE altogether, rather than as a cue to bring in a red team and get a qualitative read instead.

The case that documentation-only implementation creates real liability

A publicly traded technology company disclosed in its annual report that it had implemented the NIST AI RMF as part of its AI governance program. Six months later, a generative AI feature in one of its products started producing systematically wrong outputs, affecting a measurable, documented group of customers.

What followed wasn't subtle. Customer complaints piled up, press coverage followed, and the SEC opened an inquiry into whether the company's earlier AI risk disclosures had accurately described the state of its governance program at the time it made them.

The gap this exposed lines up exactly with the distinction flagged back in the GOVERN section. GOVERN 1.1 asks that policies exist. It doesn't guarantee anyone follows them. MANAGE 1.1 asks that a risk treatment plan exist. It doesn't guarantee that plan gets executed when something actually breaks. Closing that gap takes operational work, not another round of documentation sitting in a binder somewhere.

The Home365 settlement pointed at the same problem from the regulatory side, making clear that regulators are willing to act against companies that treat an AI failure sourced from a vendor as somebody else's problem to solve. The framework didn't fail in the technology company's case. It was never fully implemented in the first place. GOVERN and MAP got completed, and that completion got mistaken for the whole cycle.

For teams working in TPRM, InfoSec, Privacy, and Legal, this case matters for a specific reason. The liability exposure lives in the space between documented governance and controls that actually run day to day, and that space opens widest exactly where third-party AI systems sit, which is the subject the next section takes up directly.

Requirements for third-party and vendor AI systems under MANAGE

MANAGE 4.1 covers post-deployment monitoring, appeal and override mechanisms, decommissioning, and change management. None of that language carves out an exception for vendor-supplied systems. It applies to a purchased model exactly as much as it applies to something built in-house.

The third-party AI surface is bigger than most inventories capture: vendor-supplied models, productivity apps wired into corporate identity systems, generative AI services with access to sensitive company data, plugins and connectors bolted onto existing tools, and open-source components buried inside a vendor's product that nobody outside that vendor has fully audited.

The 2026 Black Kite Third-Party Breach Report found that for every vendor breached, an average of 5.28 downstream organizations got compromised alongside it, the highest cascading impact on record. The median gap between a breach happening and it becoming public was 117 days. In a growing share of these incidents, an AI tool sat at the center of the breach, operating without real visibility or governance on either side of the vendor relationship. A 117-day disclosure gap means an organization can be sitting exposed for nearly four months without knowing it, and a once-a-year vendor questionnaire simply can't catch a problem moving that fast.

GOVERN 2 requires accountability structures for how third-party AI gets introduced into the business. Most vendor risk programs, though, were built for traditional software and static data feeds, not for models that can change their own behavior after deployment through an update, a fine-tuning pass, or plain environmental drift nobody flagged in advance. Building an AI-BOM for vendor systems helps close that gap: tracking model provenance, training data lineage, update and version history, and known vulnerabilities gives a company the raw information MANAGE controls actually need to function against a third-party asset.

NIST's own generative AI profile, AI 600-1, names several of these risks explicitly rather than leaving them implied. Direct and indirect prompt injection appears in section 2.9 as an Information Security risk. Supply chain integrity gets its own line as a Value Chain and Component Integration risk under section 2.12. Data privacy risk from generative AI services touching sensitive enterprise data gets called out too. The Treasury Department's Financial Services AI RMF, released February 19, 2026, backs this up from the regulatory side with a dedicated third-party risk section among its 230 control objectives, a signal that in regulated industries, third-party AI oversight isn't optional anymore.

Traditional TPRM checks a vendor's security posture once, at a fixed point in time, and calls it done for the year. AI-specific third-party risk doesn't work that way. It needs continuous monitoring of what a vendor's AI assets are actually doing: model changes, new plugins or agents added to a product, behavior drift visible in outputs, and vulnerabilities specific to AI components that a generic vulnerability database was never built to catch.

The AI-specific attack surfaces that generic framework mappings miss

Model inversion, prompt injection, data poisoning, and emergent behavior sit in the gaps between traditional security controls, not inside any single one of them. That's the core problem with mapping a generic cybersecurity framework onto an AI system and calling the job finished.

Take model inversion. A traditional data loss prevention tool watches for files leaving a network. It has no concept of an attacker querying a deployed model repeatedly, piecing together fragments of its training data from patterns in the outputs alone, with nothing resembling a file ever moving anywhere. The control was built for a different kind of theft.

Prompt injection breaks a different assumption. Standard input validation checks for malformed code, SQL syntax, script tags. It's not built to catch a plain-English instruction buried inside a document or a webpage that a language model reads and treats as a command. NIST's AI 600-1 profile names both direct and indirect prompt injection as security risks precisely because the traditional boundary between "data" and "instruction" collapses once a model is reading everything as potential instruction.

Data poisoning targets a phase that most security reviews don't touch at all: training. A model can be corrupted well before it's deployed, through a compromised dataset or a manipulated fine-tuning run, and the resulting behavior might not appear as an obvious anomaly. It becomes visible later, in a specific case, under a specific condition an attacker chose in advance.

Emergent behavior is the hardest of the four to pin down, because it emerges only once a system's scale or context shifts, doing something unplanned in a way no static control accounts for. It's a system doing something unplanned once its scale or its context shifts, and no static control, no matter how well-tuned, checks for a capability that didn't exist at the moment the control was configured.

One might argue this is why GOVERN, MAP, MEASURE, and MANAGE cycle continuously instead of running once and closing out. A framework can only account for a risk if someone thought to look for it, and a lot of AI-specific risk lives precisely in the places nobody thought to check yet, because the last audit was designed for a different kind of software.

Sources

  1. NIST AI RMF: Practical Implementation Guide
  2. netsolutions.com
  3. nist.gov
  4. NIST AI RMF Playbook
  5. hoodline.com
  6. attorneygeneral.gov
  7. nvlpubs.nist.gov

More in AI Security Governance