The 2026-09-03 YZ Index Smoke quick test covered 11 models, with GPT-o3 ranking first at 83.94 points. Smoke is a daily 10-question quick test suited to observing short-term signals and does not equate to Full weekly ranking conclusions.
The latest WDCD v3.1 pilot data shows Gemini 3.1 Pro ranking first with a commitment-adherence score of 97.70, while Gemini 2.5 Pro posted the largest gain at 22.2 points. Claude Sonnet 4.6 was the only model to decline, falling by 13 points.
WDCD v3.1 pilot data shows that safety and compliance is the weakest scenario on average across 11 models, with qwen3-max scoring only 2/4 and deepseek-v4-pro 2.5/4. The results reveal significant scenario-specific differences in models’ ability to retain constraints and withstand pressure.
In the WDCD v3.1 pilot phase, worst-of-3 sampling across 8 v2 anchor questions produced an average R3 integrity rate of only 72.7%, with 3 complete R3 collapses (0 points) across 110 tests. All three failures occurred on the same multi-constraint security compliance question, making R3 pressure resistance the key differentiator among models.
In the WDCD v3.1 commitment test, Gemini 3.1 Pro led all 11 evaluated models with 97.70 points, while Qwen3 Max ranked last with 70.30 — a gap of 27.4 points. Sampling followed the worst-of-3 methodology, with scores strictly applied across the S_hold, S_kbv, S_recover, and S_integrity components.
On 2026-09-02, the YZ Index Smoke quick test covered 11 models, with Doubao Pro ranking first at 95.91 points. Multiple leading models showed notable score swings that require further review.
GPT-5.5's code execution score dropped sharply from 94.50 to 69.50 in today's Smoke evaluation, while material constraint rose from 76.00 to 100.00. The main index edged down from 86.18 to 83.23 after the two dimensions offset each other.
Grok 4's code execution score in today's Smoke evaluation plunged from 94.50 to 72.70, dragging the main leaderboard down 4.2 points, while its material constraint score rose to a perfect 100.00.
The 2026-09-01 YZ Index Smoke quick test covered 11 models, with GPT-o3 ranking first at 96.98 points. Notable shifts include gains for DeepSeek V4 Pro and Gemini 2.5 Pro, while Grok 4 and GPT-5.5 posted sharp drops in code execution.
On 2026-08-31, the YZ Index Smoke quick test covered 11 models, with Grok 4 ranking first at 89.15 points. Smoke is a daily 10-question quick test suited for short-term signals, not equivalent to the Full weekly ranking conclusions.
Gemini 3.1 Pro's main score in today's Smoke evaluation fell from 96.60 to 85.83, a drop of 10.8 points, driven primarily by a 14.4-point decline in the material constraint dimension.
Grok 4's main ranking score in today's Smoke evaluation fell from 96.99 to 88.14, a single-day decline of 8.8 points, with code execution dropping from 100.00 to 90.70. The decline appears driven by question-draw fluctuation rather than genuine model degradation.
In the Smoke evaluation from August 24 to August 30, 2026, Claude Opus 4.7 rose from 82.98 points on day one to 93.08 points on the final day, a trend increase of 10.1 points, while Gemini 3.1 Pro fell 11.2 points over the week.
On 2026-08-30, the YZ Index Smoke quick test covered 11 models, with Claude Opus 4.7 ranking first with 93.08 points. Smoke is a daily 10-question quick test suited for observing short-term signals and is not equivalent to Full weekly ranking conclusions.
MLCommons' MLPerf Inference working group has launched the first end-to-end retrieval-augmented generation (RAG) inference benchmark, measuring the full pipeline from document ingestion to multi-hop question answering across two workloads.
This article explores how double-blind evaluation design protects privacy and benchmark integrity, highlighting MLCommons' first proof of concept for closed-source model evaluation and its path toward becoming an industry standard.
In today's Smoke evaluation, Claude Sonnet 4.6's code execution score plunged 22 points to 75.00, while its material constraint score surged 25.7 points to 93.30; the main leaderboard score slipped just 0.5 points to 83.24. The extreme inverse swings are attributed to small-sample question sampling volatility rather than genuine model degradation.
Claude Opus 4.7 scored 83.24 points on today's Smoke evaluation main leaderboard, down 10.3 points from yesterday's 93.54, driven by a drop in the code execution dimension from 97.00 to 75.00 points.
The YZ Index Smoke quick test on 2026-08-29 covered 11 models. Grok 4 topped the day with 96.99 points, while notable gains were seen for Gemini 3.1 Pro, GLM-4.6, DeepSeek V4 Pro, and Grok 4, and sharp declines were recorded for GPT-5.5 and several Claude models.
Pushing the Limits of Serving DeepSeek-V4-ProTianyu Zhang, Yusong Gao, Yun ZhangAugust 19, 20261. Introduction DeepSeek-V4-Pro is a 1.6-trillion-parameter Mixture-of-Experts (MoE) model released with
