Training V5 1 Results

MLCommons has released the MLPerf Training v5.1 benchmark results, the latest progress in AI model training performance evaluation. This submission covers nine core workloads, attracting participation from multiple vendors such as NVIDIA, Intel, AMD, and Google Cloud, showcasing training capabilities ranging from single nodes to large-scale clusters with thousands of GPUs.

Key Highlights and Record Holders

NVIDIA dominated most benchmarks with its DGX H100 and H200 systems:

• BERT: NVIDIA set a new record of 1,363.94 samples/s using 8x H100 SXM.
• DLRM v2.0: NVIDIA achieved 2,882.59 samples/s with 256x H100 NVL.
• ResNet-50: NVIDIA reached 1,360,368 images/s with 2,048x H100.
• Stable Diffusion XL (new benchmark): NVIDIA achieved efficient generation with 2,048x H100.
• Llama 70B FP8 (new benchmark): NVIDIA used 1,536x H100, introducing FP8 quantization training for the first time, significantly reducing computation requirements.

New Benchmarks and Technological Innovations

v5.1 introduces several cutting-edge workloads:

• GPT-J 6B: Tests training on small-to-medium generative models.
• Llama 70B FP8: Validates the practicality of low-precision training, with NVIDIA taking the lead.
• Stable Diffusion XL: Focuses on diffusion model image generation training.

Intel submitted Gaudi 3 results, performing well on Llama 70B FP8; AMD’s MI300X showed progress in benchmarks such as ResNet-50. Cloud providers like Google Cloud and CoreWeave also participated, emphasizing production-grade deployment.

Performance Trends and Implications

Compared to v5.0, results show a 20%–50% improvement in training speed, driven by NVLink interconnect, TransformerEngine optimization, and quantization techniques. MLPerf Training benchmarks are evolving toward larger models—potentially including Llama 405B in the future—driving standardization and fair comparison across AI hardware.

For full results, see the MLCommons official website.