GPT-5.5's code execution score in the Smoke evaluation fell from 100.00 to 75.00, pulling the main leaderboard down from 78.49 to 73.20. The drop appears to reflect question-sampling variance rather than a genuine capability regression, as other dimensions rose.
In today's Smoke evaluation, Grok 4's code execution score fell from 94.60 to 74.40, a 20.2-point drop, while material constraints rose from 45.00 to 75.00 and the main leaderboard score rose from 72.28 to 74.67. The opposite moves point more to question-draw volatility than systematic degradation.
The 2026-10-28 YZ Index Smoke quick test covered 13 models, with GPT-6 Luna taking first place for the day at 86.95 points. Because Smoke is a daily 10-question quick test, single-day results are best treated as short-term monitoring signals rather than final conclusions about model capability.
The 2026-10-27 YZ Index Smoke Quick Test covered 13 models, with Claude Opus 4.7 ranking first at 87.7 points. The daily small-sample results are intended as short-term monitoring signals rather than long-term conclusions.
The 2026-10-26 YZ Index Smoke quick test covered 13 models, with GPT-6.1 Sol taking the top spot for the day at 86.05 points. Smoke is a daily 10-question quick test suited to spotting short-term signals; it is not equivalent to the conclusions of the weekly Full leaderboard.
Qwen3 Max recorded missing data across all five dimensions in today's Smoke evaluation due to API failures/timeouts, triggering an automatic rerun and exclusion from this round's main leaderboard.
The 2026-10-25 YZ Index Smoke quick test covered 14 models, with Grok 4 leading at 96.65. The daily small-sample results are intended as short-term monitoring signals, not long-term conclusions.
The 2026-10-24 YZ Index Smoke quick test covered 13 models, with Claude Sonnet 4.6 taking the top spot for the day at 89.8 points. Smoke is a daily 10-question quick test suited to spotting short-term signals and is not equivalent to the conclusions of the weekly Full leaderboard.
GLM-4.6 scored 18.25 on the main leaderboard in the Run #392 Smoke quick test, with code execution at 25.00 and material constraint at just 10.00, while its integrity rating failed and its probe score was 0.00. The result points to serious weaknesses in fidelity to provided materials and citation behavior.
The 2026-10-23 YZ Index Smoke quick test covered 14 models, with Doubao Pro and GPT-6.1 Sol tied for first at 87.27. The single-day, small-sample results are intended as short-term monitoring signals rather than long-term conclusions.
In today's Smoke evaluation, Claude Sonnet 4.6's main leaderboard score fell from 80.99 to 64.13, a 16.9-point drop driven mainly by material constraints falling from 88.30 to 58.30. The 30-point single-day drop in material constraints likely reflects sample variation from the 10-question daily set, while engineering judgment and the integrity rating remained unchanged.
In today's Smoke evaluation, Gemini 3.1 Pro's main leaderboard score rose to 90.84, but material constraints fell 16.7 points while code execution rose 27 points. The shift most likely reflects random question draws rather than a structural change in model capability.
The 2026-10-22 YZ Index Smoke quick test covered 14 models, with GPT-5.5 taking the top spot for the day at 96.1 points. Smoke is a daily 10-question check suited to short-term signals rather than full weekly ranking conclusions.
The EU has selected MLPerf Training v5.1 and MLPerf Inference v5.1 as performance benchmarks for its AI Gigafactories tender, while MLCommons has opened submissions for MLPerf Endpoints v1.0 to evaluate real API-served AI services across performance, cost, and energy consumption.
MLCommons has updated its mission to focus on standardizing AI measurement and enabling informed decisions. It will continue building and operating industrial-grade benchmarks while evolving its organization and outreach.
MLPerf Endpoints evaluates inference systems as complete production endpoints under real workloads, presenting a Pareto front across concurrency levels and metrics directly tied to deployment decisions. The article explains what the benchmark measures, how it differs from other benchmarks, and how buyers should interpret its results.
MLCommons has been selected as the benchmark team for the STRIDE Ventures AI Efficiency Challenge and will use a two-year, $3.5 million NSF-supported grant to develop MLPerf Agents, a new benchmark for measuring the efficiency of agentic AI systems in completing real work.
SGLang Adds Day-0 Support for NVIDIA Nemotron 3.5 Super VLNVIDIA Nemotron TeamOctober 15, 2026Always-on AI agents must do more than generate text. They need to interpret visual information, reason acr
SGLang and Miles Add Day-0 Support for Muse Spark OpenSGLang TeamOctober 15, 2026Highlights Model: Muse Spark Open is an open-weight multimodal mixture-of-experts model. It uses an 816B-parameter text
SGLang Powers Beam in ProductionSGLang TeamOctober 16, 2026Reflection’s Beam is a 501B-parameter mixture-of-experts model with 23B active parameters per token, built for software engineering and long-
