In today's Smoke evaluation, Claude Opus 4.7's material constraint score fell from yesterday's 88.30 to 71.40, a drop of 16.9 points, while its main leaderboard score rose from 79.56 to 84.11, a net gain of 4.6 points.
Score Comparison Data
The code execution dimension rose from 72.40 to 94.50, an increase of 22.1 points; the material constraint dimension fell from 88.30 to 71.40; the engineering judgment sub-index fell from 55.60 to 50.00; and the task expression sub-index fell from 85.40 to 55.40. The integrity rating remained at pass.
Direct Reason for the Net Rise on the Main Leaderboard
The main leaderboard is weighted from only two dimensions: code execution and material constraint. A single-day gain of 22.1 points in code execution outweighed the negative impact of the 16.9-point drop in material constraint, pushing the main leaderboard upward overall. Although the material constraint decline was large, it did not offset the increase in code execution.
Analysis of Possible Causes
The Smoke evaluation uses only 10 questions per day, 2 per dimension, so day-to-day variation in question sampling is the most likely cause. If the material constraint items happened to include cases demanding greater fidelity to source material, deviations in the model's answers would directly drag down that dimension's score. If the code execution items happened to include relatively simple debugging or calculation tasks, the model's performance would easily show a large improvement. The task expression sub-index fell by as much as 30 points, further suggesting that the day's question distribution may have shifted away from the range where the model produces stable output, rather than indicating a systemic degradation in model capability.
The possibility of genuine degradation currently lacks support from consecutive multi-day data. Material constraint and task expression declined at the same time in a single day, while code execution rose sharply, showing that the model's performance differences across task types were amplified by that day's questions.
What This Means for Users
Teams that lean heavily on code execution can expect higher usability under today's data; a code execution score of 94.50 in the Smoke evaluation is already near the high range. Scenarios that depend on fidelity to source material (such as document summarization, contract review, and knowledge extraction) face risk, since the material constraint score of 71.40 is already below yesterday's level and outputs may deviate from the source text more often.
Both the engineering judgment and task expression sub-indices moved lower at the same time, suggesting that the model's consistency may fluctuate in workflows requiring complex decisions or clear task decomposition. When calling the model in production environments, developers need to add a secondary verification step for outputs related to material constraint.
Strategic Assessment
Based on single-day data, the rise on the main leaderboard was driven mainly by code execution, and the material constraint decline is more likely due to question sampling variance than to genuine model degradation. The current signal is insufficient to conclude that Claude Opus 4.7 has a systemic problem; it is recommended that the next Smoke evaluation continue tracking the two main leaderboard dimensions of material constraint and code execution. If material constraint stays below 75 for two consecutive days, a multi-day stability check should be initiated.
For enterprises choosing a model, today's data does not support removing Claude Opus 4.7 from material-sensitive scenarios, but a material verification step should be added to the workflow. The improvement in code execution performance is worth further testing in purely code-based tasks.
Data source: YZ Index | Run #423 | View raw data
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接