Mlperf Tiny V1 3 Results

The MLCommons organization has released the MLPerf Tiny v1.3 benchmark results, a key milestone for edge AI. It evaluates model performance on resource-constrained microcontrollers and embedded devices.

Mlperf Tiny V1 3 Tech

MLPerf Tiny v1.3 is the latest edge AI benchmark version from MLCommons, designed for resource-constrained devices. It introduces two new benchmarks (Image Classification and Visual Wake Words) and optimizes existing ones, with evaluation rules covering closed and open divisions.

Croissant Mcp

MLCommons has officially launched Croissant MCP, a major upgrade to the Croissant metadata format designed specifically for AI model cards. This standard, contributed by partners including LMSYS Org, aims to address the fragmentation of current model documentation.

Ailuminate Jailbreak V05

MLCommons and LMSYS Org have launched the AILuminate Jailbreak V05 benchmark, introducing complex multi-turn attack chains and roleplay prompts. Claude 3.5 Sonnet tops the ranking with 1485 Elo, followed closely by GPT-4o and Claude 3 Opus.

Training Flux1

This report focuses on the training details of Flux.1, an open-source text-to-image generation model from Black Forest Labs, revealing the entire process from data preparation to deployment optimization.

Training Llama 3 1 8b

LMSYS Org, in collaboration with MLCommons, has released a training benchmark report for the Llama 3.1 8B model. Based on MLCommons' standardized training benchmarks, the report details the entire pipeline from data processing to model convergence, providing reliable references for AI researchers and practitioners.

Iso Aus

MLCommons, in collaboration with LMSYS Org, has launched the ISO-AUS benchmark, a novel AI model evaluation framework designed for isolation-aware serving scenarios. The benchmark focuses on model isolation, resource fairness, and low-latency response in multi-tenant environments.

Training V5 1 Results

MLCommons has released the MLPerf Training v5.1 benchmark results, the latest progress in AI model training performance evaluation. The submission covers nine core workloads and includes participants such as NVIDIA, Intel, AMD, and Google Cloud, demonstrating training capabilities from single-node to thousands-of-GPU large-scale clusters.

Mlperf Client 1 5 Release

MLCommons announces the release of MLPerf Client 1.5, the latest benchmark suite for client inference scenarios, focusing on AI performance evaluation on mobile devices, laptops, and edge devices to provide more realistic testing standards.

Medperf Adds Webui Capabilities

MLCommons recently announced that its open-source privacy-preserving machine learning benchmarking platform, MedPerf, has officially added a WebUI feature. This update greatly improves usability, allowing developers to perform model evaluation and benchmarking through a browser without complex environment setup.

Vlm Inference Shopify

MLCommons released the latest VLM (Vision-Language Model) inference benchmark results, with the submission from the <strong>Shopify</strong> team attracting significant attention. Supported by LMSYS Org, the benchmark focuses on the inference performance of vision-language models in high-load e-commerce scenarios, aiming to provide standardized evaluation for production deployment.

KTransformers Accelerates SGLang's Heterogeneous Inference

KTransformers, developed by Tsinghua University's MadSys and Approaching.AI, optimizes CPU/GPU collaborative inference for sparse MoE models through AMX-optimized kernels, efficient device coordination, and expert deferral mechanisms, now integrated into SGLang for enhanced performance.

SGLang-Diffusion: Two Months of Progress

SGLang-Diffusion has achieved 2.5x performance improvements since its launch in November 2025, with support for new models, LoRA, parallel processing, and ComfyUI integration.

SGLang Pipeline Parallelism: Million-Token Context Extension and Performance Breakthroughs

SGLang launches a highly optimized Pipeline Parallelism implementation designed for ultra-long context inference challenges. Through integrated optimizations and a clean design, it achieves a 3.31x speedup in prefill throughput for DeepSeek V3 on multi-node H20 clusters, demonstrating strong scalability for trillion-parameter models.

FP4 Mixed-Precision Inference Optimization on AMD GPUs

We developed Petit, a collection of FP16/BF16 × FP4 mixed-precision GPU kernels for AMD GPUs, enabling 1.74× faster Llama 3.3 70B inference on existing MI250/MI300 hardware without upgrades.

SGLang Achieves Deterministic Inference and Reproducible RL Training

SGLang implements fully deterministic inference with only 34.35% performance overhead and enables 100% reproducible RL training in collaboration with slime, providing reliable solutions for rigorous scientific experiments.

GB200 NVL72 Deployment DeepSeek Optimization (Part 2): 3.8x Prefill and 4.8x Decode Throughput

The SGLang team shares their optimization progress on DeepSeek V3/R1 inference performance using GB200 NVL72, achieving 26,156 input tokens/s for prefill and 13,386 output tokens/s for decode per NVIDIA Blackwell GPU through techniques like FP8 attention, NVFP4 MoE, and large-scale expert parallelism.

Partnering with SGLang: Best Practices for Efficiently Deploying DeepSeek-R1 on H20-96G

This article presents comprehensive optimization strategies for deploying DeepSeek-R1 on H20 GPUs, achieving state-of-the-art performance of 16.5k input tokens/s and 5.7k output tokens/s per node through hardware-aware parallelization, kernel optimizations, and advanced scheduling techniques.

PD-Multiplexing: A New Paradigm for High-Goodput LLM Serving Driven by GreenContext

This article introduces PD-Multiplexing, a new serving paradigm in SGLang that leverages NVIDIA's GreenContext technology to achieve higher goodput for LLM services through efficient intra-GPU resource sharing between prefill and decode phases.

SGLang Supports DeepSeek V3.2 Sparse Attention Mechanism from Day 0

SGLang announces Day 0 support for DeepSeek-V3.2, implementing DeepSeek Sparse Attention (DSA) mechanism that significantly improves training and inference efficiency, especially in long-context scenarios.