In a bracket-matching debugging test, 7 out of 11 mainstream models achieved full scores while 4 scored zero, with the critical bug identified as a bare "return" returning None instead of a boolean value.
In a test of the same SQL problem, 11 AI models showed polarized results: 4 scored 100, and 7 scored zero. The core differences lie in self-join deduplication logic, time difference calculation function selection, and the placement of the status condition.
Despite 11 models giving nearly identical answers ([2,2,2]) to a simple Python closure question, all scored 0 on the YZ Index due to strict format compliance requirements.
In the v6 evaluation, GPT-o3's main score rose from 75.86 to 82.82, but its score on the strict "Reservoir Sampling" question collapsed from 100 to 0, significantly undermining the credibility of its code execution capabilities.
In the v6 evaluation, Claude Sonnet 4.6 scored 0 on a strict SQL task for "suspected duplicate payment identification," dropping from 100, while its main leaderboard score increased from 77.98 to 87.24. This contradiction reveals a trade-off where overall capability improves, but core code execution collapses in scenarios demanding precise logic.
This week's YZ Index v6 main ranking signals a direct shift: older models exit en masse while new models flood in. Among the seven debut models, Qwen3 Max, Grok 4, and 文心一言 4.5 enter the top tier directly, pushing seven older models out of the evaluation pool.
In today's lightweight evaluation by Smoke, Claude Opus 4.7 and GPT-5.5 tied for first on the main leaderboard with 92.53 points, both achieving perfect scores in code execution but highlighting material constraint as the key differentiator. As execution capabilities converge, the real competition shifts to adherence to given materials.
This week's 7-day Smoke test reveals GPT-5.5's execution score plummeting while two Claude models stage a dramatic reversal, though stability remains a concern. The data also highlights volatility and integrity rating fluctuations across multiple models.
The results of the Smoke Lite evaluation on June 5, 2026, show that 9 out of 11 models tied at 77.5 on the main leaderboard, forming a rare tie. Their common feature is that all scored a perfect 100 on the Code Execution dimension, but only 50 on the Material Constraint dimension.
Smoke's quick test results today clearly show that the code execution dimension is nearly saturated. Ten out of eleven models scored 100, while GPT-5.5 dropped to 50, dragging its main leaderboard score down to 59.99.
Run #141 data shows that Grok 4 improved by 10.8 points in a single round, GPT-5.5 improved by 9.2 points, while Qwen3 Max plummeted by 10.8 points. The divergence in adherence capabilities has become clearly visible.
The most brutal finding of the WDCD compliance test is that resource constraints crippled all models, with an average score of only 1.7 across 11 models, far below the other four scenarios.
The core findings of the WDCD three-round test are clear: nearly all models scored high in the constraint establishment phase, but after two rounds of interference, over 60% of models completely abandoned their original commitments under direct pressure.
The WDCD commitment test reveals model weaknesses through a three-round dialogue design. Only three out of eleven models scored 70, with Ernie Bot 4.5 forming a clear gap at 50.
In today's Smoke quick test results, Claude Opus 4.7, Claude Sonnet 4.6, and GPT-5.5 all scored 87.76 on the main leaderboard, tying for first place. The core reason is that all three achieved a perfect 100 in code execution, while scoring 72.8 in the materials constraint dimension, triggering a warn signal.
The most direct finding from today's Smoke lightweight benchmark is that code execution ability no longer differentiates the top seven models, as all scored 100, making rankings entirely determined by material constraint scores.
Smoke's lightweight evaluation completed 10-question quick test on 11 mainstream models. Claude Sonnet 4.6 scored 99.78 points, while the GPT series collectively stuck at 74 points, highlighting the polarization in AI capabilities.
In the latest WDCD cycle, all 11 evaluated models show improvement in compliance ability, with the top five all rising and none declining. Gemini 3.1 Pro leaps into the top three with a +14.2 point gain, signaling a major shift in the competitive landscape.
The WDCD compliance test evaluates model stability under real enterprise constraints through three rounds of dialogue. The resource limitation scenario scored the lowest overall, becoming a common "stumbling block" for all 11 models.
Eleven mainstream models showed a clear degradation trajectory in the three-round WDCD test: nearly all confirmed constraints in R1, maintained 93% resistance after R2 interference, but average integrity rate dropped to only 30.5% in R3, with 200 tests directly hitting zero.
