Open, peer-reviewed measurement for today’s AI buyers: MLPerf in the EU’s AI Gigafactories tender, and Endpoints v1.0 opens for submissions

As AI accelerates adoption across industries, enterprises and government procurement teams are no longer focused only on “how fast a chip is.” They want to know: what performance can a deployed AI service deliver? What does it cost under a given workload? How much energy does it consume during operation? Since 2018, MLPerf has been an important standard for measuring AI system performance, and it is now expanding further to answer the more complex questions facing AI buyers.

MLCommons announced two key developments. First, MLPerf has been selected as the benchmark for measuring training and inference performance in the EU’s AI Gigafactories tender; second, MLPerf Endpoints v1.0 is now open for submissions, marking MLPerf’s shift from traditional benchmarking toward standardized evaluation of real AI service performance, cost, and energy consumption.

Both updates point to the same trend: industry, government agencies, and AI buyers are increasingly using open, standardized, peer-reviewed measurement results as an important basis for AI adoption and procurement decisions.

MLPerf Becomes the Performance Benchmark for the EU AI Gigafactories Tender

In July this year, the EU launched the AI Gigafactories tender program, aiming to build up to seven large-scale AI computing facilities. The program will receive up to €10 billion in funding from the EU and member states and is expected to catalyze more than €30 billion in total investment.

The tender is being procured jointly by the EuroHPC Joint Undertaking and national contracting authorities, with a bid deadline of November 12, 2026.

The tender documents explicitly list MLPerf as the standard for measuring bidders’ technical performance. Bids will be scored based on the MLPerf Training v5.1 and MLPerf Inference v5.1 benchmarks.

For both the training and inference test suites, the tender documents list several models from MLPerf standard workloads, including:

• Large models on the training side, such as Flux.1 and Llama;
• Inference-side models, such as Llama 3.1 405B, DeepSeek R1, and SDXL 1.0.

The performance information the EU requires bidders to submit is directly based on MLPerf’s standard metrics, including time-to-train in training scenarios and time to first token and time per output token in inference scenarios. Bidders also need to provide additional details such as accelerator utilization.

Notably, bidders must report not only test results for first-phase delivery capability but also performance results corresponding to second-phase expansion plans. This means the EU is focused not only on currently deliverable capability but also on verifiable performance after future expansion.

For AI infrastructure procurement involving public funding on this scale, procurement teams need comparable and trustworthy performance claims. MLCommons believes that open, peer-reviewed results are key to supporting such decisions.

“MLPerf is designed to help buyers make more informed decisions when deploying AI capabilities. We are honored to see our work guiding Europe’s future in building AI leadership and empowering citizens and businesses.” — David Kanter, Head of MLPerf

MLPerf Endpoints v1.0 Opens for Submissions

Another important development is that MLPerf Endpoints v1.0 is now open for submissions. Unlike traditional benchmarks, MLPerf Endpoints measures AI as it is actually used: applications call models through real-time APIs, and the system under test is essentially a URL accessed by an Endpoints client through a standard interface.

This design allows buyers to evaluate different types of service providers under the same standard, including hardware vendors, hyperscalers, neoclouds, and managed API providers. Whether the underlying layer is bare metal, managed cloud services, or a containerized software stack, all can be compared within the same framework.

From a Single Peak to a Complete Performance Profile

MLPerf Endpoints does not provide only a single peak-performance number; it scans system behavior across a range of load conditions and outputs a complete performance profile. The results cover multiple dimensions:

• concurrency: ability to handle concurrent workloads;
• interactivity: interactive experience measured in tokens per second per user;
• system throughput: overall system throughput;
• time to first token at the 90th percentile, or TTFT P90.

These metrics help buyers match validated operating points with their own application scenarios and service-level objectives. For example, for chatbots, TTFT P90 may directly affect perceived user latency; for batch generation tasks, system throughput and cost structure may be more important.

Mandatory Provisioned Power Measurement Introduced for the First Time

Within the MLPerf family, Endpoints 1.0 introduces mandatory provisioned power measurement for the first time. This reflects MLCommons’ emphasis on energy efficiency and allows buyers to compare the energy-efficiency and cost profiles of different solutions more clearly.

Each test result is displayed as a pareto frontier in the LoadStair visualizer, showing when a system can remain stable under different loads and when it encounters a performance cliff. This is especially important for determining the safe operating range of real online services.

v1.0 Covers Generative AI and Agentic Coding Workloads

MLPerf Endpoints v1.0 builds on the v0.7 baseline released in July this year and covers multiple types of generative AI workloads, including models from the DeepSeek-R1, GPT-OSS 120B, and Llama families.

In addition, v1.0 adds an agentic coding workload, using models including DeepSeek v4 Pro, Kimi K3, and Qwen 3.6. This reflects how AI evaluation is expanding from pure text generation to more complex agentic coding and tool-use scenarios.

Rolling Submissions: New Chips and Optimizations Need Not Wait for Fixed Cycles

MLPerf Endpoints now supports rolling submissions. This means member organizations can submit and publish peer-reviewed results immediately after releasing new chips or new software optimizations, without waiting for fixed rounds.

However, all results must still go through MLPerf’s peer-review and audit process to ensure credibility and comparability. MLCommons expects the first batch of Endpoints v1.0 results to be published in November.

For members preparing submissions, MLCommons has provided submission guidelines and rules at mlcommons.org/benchmarks/endpoints. Organizations that are not yet members can also participate in the work through MLCommons.

Conclusion

From the EU’s adoption of MLPerf as the benchmark for its AI Gigafactories tender to the opening of submissions for MLPerf Endpoints v1.0 for real API services, AI infrastructure evaluation is entering a new phase that places greater emphasis on actual deployment performance, energy consumption, cost, and auditable results. For AI buyers, such open benchmarks are expected to reduce procurement uncertainty; for suppliers, they will also become an important common language for demonstrating technical capability.