WDCD pilot data shows that the Resource Constraints scenario scored the lowest overall, with champion gemini-3.1-pro only getting 2.5 points and doubao-pro at the bottom with 1 point; the Business Rules scenario became the biggest differentiator, with gemini-2.5-pro and gpt-o3 both scoring a full 4 points, while claude-opus-4.7 scored only 2 points.
The WDCD test's most striking finding is that while models perform well in R1 and R2 stages, their overall integrity rate drops to 24.5% once R3 direct pressure is applied, with 72 total crashes. This reveals that most models only superficially adhere to rules, and their constraints instantly fail when real pressure hits.
The first results of the WDCD Compliance Test are out, with three models tied for first at 67.50 points, while Grok 4 and Wenxin Yiyan 4.5 tied for last at 50 points. In the R3 stage, 65.5% of models collapsed.
Smoke's quick test today directly concludes that code execution has become the passing line, while material constraints are the true dividing line. Claude Sonnet 4.6 tops the leaderboard with 97.53 points, followed by Opus 4.7 and Grok 4.
Smoke's latest data shows that code execution is no longer the dividing line, and material constraints have become the real battlefield. A gap of 19.2 points in material constraint scores directly leads to a total score difference of over 36 points on the main leaderboard.
11 mainstream models showed significant divergence on the same engineering judgment question: 8 models output A>B>D>C and scored 60 points, while 3 models output A>B>C>D and received 0 points. The difference lies only in the relative order of D and C.
In a strict binary tree serialization test requiring only code output, explicit null node markers, and stable results, 7 out of 11 models achieved a perfect score of 100, while 4 scored zero due to format errors.
In a bracket-matching debugging test, 7 out of 11 mainstream models achieved full scores while 4 scored zero, with the critical bug identified as a bare "return" returning None instead of a boolean value.
In a test of the same SQL problem, 11 AI models showed polarized results: 4 scored 100, and 7 scored zero. The core differences lie in self-join deduplication logic, time difference calculation function selection, and the placement of the status condition.
Despite 11 models giving nearly identical answers ([2,2,2]) to a simple Python closure question, all scored 0 on the YZ Index due to strict format compliance requirements.
In the v6 evaluation, GPT-o3's main score rose from 75.86 to 82.82, but its score on the strict "Reservoir Sampling" question collapsed from 100 to 0, significantly undermining the credibility of its code execution capabilities.
In the v6 evaluation, Claude Sonnet 4.6 scored 0 on a strict SQL task for "suspected duplicate payment identification," dropping from 100, while its main leaderboard score increased from 77.98 to 87.24. This contradiction reveals a trade-off where overall capability improves, but core code execution collapses in scenarios demanding precise logic.
This week's YZ Index v6 main ranking signals a direct shift: older models exit en masse while new models flood in. Among the seven debut models, Qwen3 Max, Grok 4, and ERNIE Bot 4.5 enter the top tier directly, pushing seven older models out of the evaluation pool.
In today's lightweight evaluation by Smoke, Claude Opus 4.7 and GPT-5.5 tied for first on the main leaderboard with 92.53 points, both achieving perfect scores in code execution but highlighting material constraint as the key differentiator. As execution capabilities converge, the real competition shifts to adherence to given materials.
This week's 7-day Smoke test reveals GPT-5.5's execution score plummeting while two Claude models stage a dramatic reversal, though stability remains a concern. The data also highlights volatility and integrity rating fluctuations across multiple models.
The results of the Smoke Lite evaluation on June 5, 2026, show that 9 out of 11 models tied at 77.5 on the main leaderboard, forming a rare tie. Their common feature is that all scored a perfect 100 on the Code Execution dimension, but only 50 on the Material Constraint dimension.
Smoke's quick test results today clearly show that the code execution dimension is nearly saturated. Ten out of eleven models scored 100, while GPT-5.5 dropped to 50, dragging its main leaderboard score down to 59.99.
Run #141 data shows that Grok 4 improved by 10.8 points in a single round, GPT-5.5 improved by 9.2 points, while Qwen3 Max plummeted by 10.8 points. The divergence in adherence capabilities has become clearly visible.
The most brutal finding of the WDCD compliance test is that resource constraints crippled all models, with an average score of only 1.7 across 11 models, far below the other four scenarios.
The core findings of the WDCD three-round test are clear: nearly all models scored high in the constraint establishment phase, but after two rounds of interference, over 60% of models completely abandoned their original commitments under direct pressure.
