Assessing & Managing AI Risk for Business and Commercial Deployments: A business owner’s guide to a cost you can measure, price, and manage

Teams responsible for AI risk are caught between two goals: protecting the enterprise from harm while supporting faster, broader AI adoption, often without additional resources. MLCommons believes this is where AI governance proves its value: connecting risk management with business priorities, company values, and auditable evidence.

AI risk is already on the corporate books

Mature organizations already have a closed-loop risk management process: identify risk exposures, estimate expected losses, allocate controls according to risk level, and report residual risk to accountable management. Insurers price risk, auditors test it, and boards ultimately sign off.

Enterprises are no strangers to the cost of security incidents, and AI is amplifying those costs. IBM’s 2025 Cost of a Data Breach report shows the global average cost of a data breach is $4.44 million, while the United States reached a record $10.22 million. The same research also found that AI adoption has outpaced AI oversight: among organizations that suffered a breach, 63% had no AI governance policy or were still developing one; organizations with extensive “shadow AI” use also faced an average increase of about $670,000 in data breach costs.

Regulatory consequences are equally hard to ignore. Even before generative AI emerged, regimes such as GDPR and CCPA had already imposed strict requirements on automated decision-making that AI may involve. Penalties under the EU AI Act are even more deterring, reaching up to €35 million or 7% of global turnover, making proactive AI risk management an unavoidable issue for enterprises.

The good news is that managing the risk of deployed AI systems does not require an entirely new philosophy. Enterprises only need to demand sufficiently reliable evidence, just as they do for other operational risks, and adopt new measurement tools suited to AI scenarios.

Lessons from aviation: what can be measured can be trusted

The reward for effectively managing new risks is not just avoiding losses; it can also shape the healthy development of an entire industry. In 1959, U.S. commercial aviation had about 40 fatal accidents per million takeoffs; in less than a decade, that number fell to fewer than 2, and over the following half-century it continued to roughly halve every decade. Aviation became safe because the industry built a continuously iterating risk management machine.

Measurement made flying safe enough to earn trust and insurable at scale. Today, billions of passengers each year, along with lenders and insurers supporting more than 30,000 commercial aircraft, regard flying as a routine activity.

The era of “grading your own homework” is ending

AI risk management cannot do without trusted measurement. To build trust, organizations need independent, transparent assessment mechanisms. Pharmaceutical companies do not adjudicate their own clinical trial results, and public companies do not audit their own financial statements.

But in the AI market, vendors still often set their own benchmarks, run tests themselves, and publish the results. An AI code review case showed that one vendor’s self-test claimed an 82% bug capture rate; a competitor’s retest on the same codebase produced 45%; and when a lab with no vested interest established an independent benchmark, no vendor exceeded 63%. The product did not change; what changed was who was refereeing.

AILuminate: making vendors stop writing their own exams

MLCommons’ AILuminate® was designed precisely to fill this gap. Released in December 2024, it is a set of independent, consensus-governed benchmarks for evaluating the reliability of AI systems in avoiding harmful responses.

AILuminate was built by working groups from AI companies, academia, and civil society, and protects test integrity through private test sets and independent evaluators. In other words, vendors cannot write their own questions, cannot see the questions in advance, and cannot control the scoring.

How can governance teams use these scores?

• Identify risk concentrations: Model scores by harm category can show where risk exposure is concentrated in similar use cases, providing source material for the risk register.
• Guide controls: If a system’s scores drop significantly under adversarial pressure, this is not a vague instruction to “be careful,” but evidence-supported guidance: guardrails need to be built at the system layer.
• Test real deployment configurations: Buyers should use the same testing methods as vendors to assess the configurations they actually deploy. Base model scores are like a car’s factory fuel-economy label; they describe the delivered state, not the fuel economy you actually get on your commute.

Scores before and after deployment should both enter the risk register: quantifiable, comparable, auditable. These scores are not only evidence for external audiences; they can also serve as a feedback mechanism for continuous improvement and detecting regressions.

The direction of the AI assurance chain is already clear. In late 2025, major insurers asked U.S. regulators to allow them to exclude AI-related liabilities from corporate policies. This was not because AI risk is unmanageable, but because measurement methods are lacking; risk that cannot be measured is also difficult to underwrite.

You must test the system you actually deploy

Base model scores can only demonstrate how a model performed when delivered by its developer; they cannot explain its actual operating state in an enterprise environment. Most organizations do not deploy bare models directly; instead, they connect models to prompts, internal data, tool calls, and sometimes have them work as agents. It is this “harnessing” that makes models useful, but it can also make vendors’ built-in safety measures fragile.

This evolution of assurance mechanisms can be seen in efforts such as the Agent Reliability Profile. The project comes from the MLCommons Financial Services Working Group and was a finalist in this year’s CDIR hackathon. It is an attestation document for a single agent, recording a bounded, falsifiable claim: that a system can operate reliably within a specific autonomy tier, a specific operational design domain, and a specific control envelope.

This is not a vague feeling about whether an agent is “trustworthy,” but a concrete claim that an auditor or insurer can verify.

Manage AI risk like any other risk

MLCommons’ core point is that AI risk can be identified, priced, managed proportionally, and reported, just as enterprises have managed credit risk, foreign exchange exposure, and workplace safety for decades. Doing so is not a brake on AI adoption, but a prerequisite for responsibly capturing AI’s benefits.

AI risk is already on the corporate books. The real question is whether enterprises are willing to start measuring it.

MLCommons says AILuminate is intended to help organizations address this problem and support teams in measuring AI risk. This article is adapted from the longer white paper How to Think About AI Risk: A business owner’s guide to a cost you can already measure, price, and manage.

Further reading

• McKinsey, The State of AI (2025) and The AI reckoning: How boards can evolve (2026)
• PwC, 2026 Annual Corporate Directors Survey
• IBM, Cost of a Data Breach (2025)
• Center for Democracy & Technology, Getting Third-Party AI Assessment Right (2026)
• NIST, AI Risk Management Framework
• MLCommons, AILuminate v1.1 and Jailbreak Benchmark v1.0, arXiv:2610.02827 (June 2026)