GPT-5.5's code execution score dropped sharply from 94.50 to 69.50 in today's Smoke evaluation, while material constraint rose from 76.00 to 100.00. The main index edged down from 86.18 to 83.23 after the two dimensions offset each other.
Grok 4's code execution score in today's Smoke evaluation plunged from 94.50 to 72.70, dragging the main leaderboard down 4.2 points, while its material constraint score rose to a perfect 100.00.
The 2026-09-01 YZ Index Smoke quick test covered 11 models, with GPT-o3 ranking first at 96.98 points. Notable shifts include gains for DeepSeek V4 Pro and Gemini 2.5 Pro, while Grok 4 and GPT-5.5 posted sharp drops in code execution.
On 2026-08-31, the YZ Index Smoke quick test covered 11 models, with Grok 4 ranking first at 89.15 points. Smoke is a daily 10-question quick test suited for short-term signals, not equivalent to the Full weekly ranking conclusions.
Gemini 3.1 Pro's main score in today's Smoke evaluation fell from 96.60 to 85.83, a drop of 10.8 points, driven primarily by a 14.4-point decline in the material constraint dimension.
Grok 4's main ranking score in today's Smoke evaluation fell from 96.99 to 88.14, a single-day decline of 8.8 points, with code execution dropping from 100.00 to 90.70. The decline appears driven by question-draw fluctuation rather than genuine model degradation.
In the Smoke evaluation from August 24 to August 30, 2026, Claude Opus 4.7 rose from 82.98 points on day one to 93.08 points on the final day, a trend increase of 10.1 points, while Gemini 3.1 Pro fell 11.2 points over the week.
On 2026-08-30, the YZ Index Smoke quick test covered 11 models, with Claude Opus 4.7 ranking first with 93.08 points. Smoke is a daily 10-question quick test suited for observing short-term signals and is not equivalent to Full weekly ranking conclusions.
MLCommons' MLPerf Inference working group has launched the first end-to-end retrieval-augmented generation (RAG) inference benchmark, measuring the full pipeline from document ingestion to multi-hop question answering across two workloads.
This article explores how double-blind evaluation design protects privacy and benchmark integrity, highlighting MLCommons' first proof of concept for closed-source model evaluation and its path toward becoming an industry standard.
In today's Smoke evaluation, Claude Sonnet 4.6's code execution score plunged 22 points to 75.00, while its material constraint score surged 25.7 points to 93.30; the main leaderboard score slipped just 0.5 points to 83.24. The extreme inverse swings are attributed to small-sample question sampling volatility rather than genuine model degradation.
Claude Opus 4.7 scored 83.24 points on today's Smoke evaluation main leaderboard, down 10.3 points from yesterday's 93.54, driven by a drop in the code execution dimension from 97.00 to 75.00 points.
The YZ Index Smoke quick test on 2026-08-29 covered 11 models. Grok 4 topped the day with 96.99 points, while notable gains were seen for Gemini 3.1 Pro, GLM-4.6, DeepSeek V4 Pro, and Grok 4, and sharp declines were recorded for GPT-5.5 and several Claude models.
Pushing the Limits of Serving DeepSeek-V4-ProTianyu Zhang, Yusong Gao, Yun ZhangAugust 19, 20261. Introduction DeepSeek-V4-Pro is a 1.6-trillion-parameter Mixture-of-Experts (MoE) model released with
Mooncake for Miles: From Fragmented Rollout Data to Efficient Bulk I/OMooncake communityAugust 20, 2026Rollout Data in Disaggregated RL Systems Reinforcement learning for large language models combine
Fast Engine Recovery: Sub-Second Engine Restart for SGLang via Weight Cache DaemonAnt Ling Infra Team (Ant Group), Alibaba, SGLang TeamAugust 21, 2026TL;DR Nowadays, State-of-the-Art (SOTA) models are
Chasing the Batch-1 Floor: Ling-3.0-flash Speculative Decode on BlackwellRadixArk SGLang Team, Ant Ling Infra TeamAugust 21, 2026Batch-1 decode keeps getting more important. Xiaomi MiMo, for example,
Qwen3.8-Flash-Next: Day-0 Support in SGLangSGLang TeamAugust 26, 2026Introduction Today, the Qwen team open-sourced Qwen3.8-Flash-Next, a multimodal MoE model and an early preview of the Qwen4 archite
MiniMax-H3 on 8×H200: 1.95× Lossless, Up to 6.24× at 0.76–0.91 SSIMSGLang Diffusion Team, Cache-DiT Team, NVIDIA, Ant GroupAugust 27, 2026TL;DR We benchmarked MiniMax-H3 video generation on 8× NVIDIA
Infer-forge: Harness, Loop, and Graph Engineering Around SGLangTianyu Zhang, Hanlin Gao, Yusong Gao, Yun ZhangAugust 28, 20261. Introduction Inference optimization may look local in code, but its vali
