2025 Mlc Rising Stars

2025 MLC Rising Stars List Released

MLCommons recently unveiled the 2025 MLC Rising Stars list, a special recognition for first-time submission results in the MLPerf Inference v5.0 benchmark. These systems, employing the MLC (ML Compiler) framework, stood out across various AI inference tasks, demonstrating the potential of emerging hardware and software optimizations.

List Highlights

• NVIDIA H200 achieved the highest throughput in LLM inference tasks (e.g., Llama 3.1 405B) using the SGLang runtime, reaching over 1500+ tokens/s in server-side offline scenarios.
• AMD MI300X performed exceptionally in image generation tasks (Stable Diffusion XL), earning a spot on the Rising Stars list with its first submission.
• Other Rising Stars include systems based on Intel Gaudi3 and custom accelerators, optimizing latency and energy efficiency in edge and server scenarios.

These results are based on the standard MLPerf benchmark suite, including classic models like GPTJ and BERT, as well as emerging diffusion models and multimodal tasks.

Role of the MLC Framework

MLC, as an open-source compiler supporting the TVM backend, compiles models into efficient machine code for various backends such as WebGPU and CUDA. This list highlights its advantages in production deployment, particularly with quantized models (e.g., INT4/INT8), achieving acceleration of up to 2-3x.

Significance for the AI Industry

The Rising Stars list encourages more vendors to participate in open-source benchmarks, fostering hardware diversity and software ecosystem development. Organizations like LMSYS Org have also stated that they will continue tracking the performance of these systems in real-world scenarios such as the Chatbot Arena. For the full results, see the official link.