Fresh Benchmarks, Reliable Scores: Introducing Continuous Prompt Stewardship for AI Risk Evaluation

The AI industry launches new frontier models every few months, each more powerful than the last, while also shifting the risk landscape that regulators, enterprises, and the public need to assess. Yet the benchmarks used to measure these risks do not automatically update. A benchmark designed for last year's model may fail to diagnose this year's model.

This is the core challenge for AI evaluation: assessment tools must keep pace with technological development. If they fail to do so, the result is not a dramatic failure but a silent erosion. Scores are still generated, grades are still assigned, but these numbers gradually lose their ability to reflect real-world risks, leaving organizations that rely on them to operate based on outdated signals.

Since the launch of LLM chatbots, AI benchmarks have proliferated rapidly. However, few benchmarks have mechanisms to address the fundamental issue of "benchmark freshness." Another complicating factor is that benchmarks typically make evaluation datasets public, allowing model developers to train directly on test data. Although many foundation model organizations have policies against this practice, even these organizations struggle to ensure test data does not mix into the ever-growing training dataset. When models train on benchmarks, scores reflect memorization rather than genuine risk management or capability. BenchRisk, an independent framework for assessing benchmark quality across 57 failure modes, quantifies this problem: among 26 AI benchmarks evaluated, the median longevity score is only 5/100. These benchmarks saturate, become compromised, or simply stop differentiating systems. AILuminate, the first benchmark developed by the AI Risk and Reliability (AIRR) Working Group of MLCommons, is specifically designed to resist this pattern. Its v1.0 prompt dataset includes 24,000 human-crafted prompts covering 12 harm categories. It is privately managed, incorporates reserved prompt sets for rotation, and achieves the highest composite score among all 26 benchmarks, including a longevity score of 75. However, while AILuminate's longevity may be superior to comparable benchmarks, it still degrades over time. Ensuring AILuminate continues to provide reliable real-world information means the benchmark itself requires maintenance.

A key component of AILuminate's long-term value is the operational infrastructure used to maintain benchmark freshness: what we call the Continuous Prompt Stewardship System. In this system, "Continuous" means that prompt refreshment is driven by technical requirements based on quantitative measurements of prompt performance, not waiting for organizational bandwidth or calendar cycles. "Stewardship" signifies the custodial management of a shared community resource, with duties of care, transparency, and accountability. This reflects MLCommons' mission to achieve "Better AI for Everyone." MLCommons' multi-stakeholder community spans industry, academia, government, civil society, and the broader public. Our prompt stewardship infrastructure is designed to maintain the integrity of the benchmark on their behalf.

What Maintaining Benchmark Freshness Requires

This sounds simple in principle, but requires solving several interrelated issues simultaneously. You need quality metrics for each prompt to detect obsolescence. You need reserve prompts ready for rotation. You need quality metrics for new prompts, and metrics for the entire prompt dataset, to ensure comprehensive coverage of harm categories and sufficient diversity to resist overfitting. These metrics need a solid scientific foundation, not just editorial judgment. You need a sufficiently broad pipeline of contributors to generate diverse, appropriately representative, and natural prompts at the speed required by the benchmark. This contributor pipeline must include rigorous quality control to withstand the scrutiny that industry-standard benchmarks attract. Moreover, all of this needs to be documented and auditable, because the credibility of every benchmark run by MLCommons ultimately depends on the integrity of the prompts that generate it. To meet these requirements, the Prompt Stewardship System makes the following changes to how AILuminate manages its prompt dataset.

Refresh cadence driven by prompt metrics. Prompt rotation will be driven by empirical performance, such as observed decrease in discriminative power, ceiling effects, emerging inter-prompt correlations, etc. We employ measurement methods based on psychometric principles, particularly Item Response Theory, a measurement framework used in standardized tests from the SAT to medical licensing exams.

Closed-loop dataset rebalancing. Whenever prompts are added or retired, the system recomputes dataset-level metrics, such as coverage balance across all 12 harm categories, difficulty distribution, and language diversity. Gaps identified through rebalancing (e.g., reduced coverage in a harm category, a difficulty band becoming sparse, etc.) generate specifications and requirements for the next prompt generation cycle. Rebalancing closes the loop between retirement and generation. Even as individual prompts rotate, the overall measurement properties of the entire dataset are preserved.

Community-driven contributor model. The v1.0 prompts were produced by contracted vendors following specifications, similar to the "cathedral" model described by Eric Raymond in his seminal paper on open-source development. It effectively delivered the initial dataset. But it concentrated expertise in a few organizations and limited the speed and diversity of prompt production. The Prompt Stewardship System shifts to an open collaborative model that Raymond likened to a "bazaar," broadening the pool of authors to include MLCommons staff, volunteers from member organizations, certified public contributors, and hired experts. This shift increases scale and quality, as a diverse contributor base produces prompts with more natural variation in style, vocabulary, and cultural framing. However, an open contribution model only works if quality control scales with it. Wikimedia produces reference-quality knowledge at a scale unmatched by contract labor, not because anyone can edit anything, but because of tiered trust levels and shared standards. The Prompt Stewardship System applies the same principle: each contributor progresses through a documented qualification pathway, with their status recorded at each step. The result is not a vague assertion of "expert authorship," but documented, quantitative evidence that each contributor meets the same standards.

Dual-path review for borderline cases. AILuminate uses an "LLM-as-judge" approach. Scoring responses with a dedicated evaluation model is highly scalable, but every LLM-as-judge has limitations. When prompts are ambiguous, culturally nuanced, or test difficult risk boundaries, the evaluator may struggle to produce high-confidence scores. Across the industry, benchmarks lack infrastructure to handle these items; they are either included with noisy scores or quietly excluded, with human review filling the gap in neither case. We consider this practice backward. Prompts that are evaluator-incompatible often test the most important and hardest boundaries—cases where human judgment matters most. The Prompt Stewardship System routes these cases to qualified human reviewers, building ground truth in the areas that are hardest to measure.

Human ground truth density. Most benchmarks, including AILuminate v1.0, conduct human review when needed, relying on individual judgment about when and where human oversight is necessary. This approach is reasonable, but it differs from treating human review as a measurable, trackable attribute of the benchmark. This requires a ground truth density metric, a dataset-level measure of how much of the prompt set has been validated by qualified human reviewers, tracked across harm categories. This metric transforms human oversight from an ad hoc practice into a reportable, tiered coverage target. MLCommons can then make quantitative statements about the level of human review behind benchmark results.

Whitelisted testing channel. Prompts designed to probe AI risks are inherently intended to elicit harmful responses. Submitting thousands of such prompts through standard LLM API access triggers the abuse detection mechanisms that providers use to protect their platforms. The Prompt Stewardship System operates through a whitelisted channel: direct agreements with AI system providers authorizing the submission of evaluation prompts. No academic benchmark maintains such infrastructure, which is a core reason why institutional benchmarks at this scale require an independent organization with established provider relationships.

Auditable provenance. Each prompt will carry documentation including who wrote it, when, under what methodology, and why it was included. As a 501(c)(6) non-profit that produces industry-standard benchmarks, MLCommons expects benchmark decisions to withstand external scrutiny. The provenance framework ensures that prompt selection is defensible—not only technically sound, but transparently so.

Why This Matters Beyond MLCommons

Prompt stewardship addresses a problem not unique to AILuminate. Every benchmark faces similar lifecycle challenges: obsolescence, contamination risk, and more.