The patch model is breaking. AI evaluation needs a new way to disclose what it finds.

For thirty years, the security community has relied on Coordinated Vulnerability Disclosure to handle dangerous findings. This approach operates by reporting privately, fixing privately, and then disclosing details publicly. The core premise is that systems can be fixed and fixes eliminate harm.

However, this premise does not hold for AI systems. MLCommons recognized this issue when building safety and jailbreak benchmarks, and sees it as a core governance challenge for frontier model evaluation.

Why the Coordinated Disclosure Model Fails for AI

Three characteristics of AI evaluation break the traditional model:

Findings are inherently dual-use

Evaluation results help defenders, regulators, and users understand system behavior, but equally guide adversaries. They reduce the cost for adversaries to locate vulnerabilities, generating an "uplift" effect. Even without exposing secret capabilities, they lower the threshold for finding capabilities. If results are publicly disclosed by default but certain categories are withheld, that withholding itself becomes a signal.

Too much feedback compromises the test

Reproducible benchmarks face a tension: developers need feedback to improve general properties, but cannot receive specific test instances. Providing exact prompts leads the model to optimize only for the test, not for genuine improvement. Only general situations should be conveyed, not specific instances, and self-justification alone is insufficient.

Released open-weight models cannot be patched

This is the key factor breaking the traditional model. A new version of an open-weight model is a new entity, not an update. Copies of old weights remain permanently in anyone's hands. If a CBRNE harm is discovered in a previous model, that harm persists indefinitely. Findings on sensitive categories need to be aggregated or uniformly hidden, avoiding becoming a targeted map.

From Principles to Standards

MLCommons has submitted its practices to ISO/IEC JTC 1/SC 42, driving the development of the ISO/IEC TS 42119-8 standard, aiming to provide a responsible disclosure framework that all evaluators can reference.

Implications for Jailbreak Benchmarks

The upcoming jailbreak benchmarks will be accompanied by a disclosure policy based on the above considerations, protecting the public from harmful uplift, maintaining evaluation integrity, and addressing risks in systems that cannot be centrally fixed. This policy is already aligned with the SC 42 standard.

MLCommons has recently formed an Agent Safety Working Group. You are welcome to join in collectively addressing AI safety challenges.