In today's Smoke evaluation, Claude Opus 4.7's main leaderboard score fell from 94.91 to 83.48, a decline of 11.4 points, while the material constraint dimension dropped from 89.30 to 70.00.
Details of the Score Changes
Code execution fell from 99.50 to 94.50, down 5 points; engineering judgment fell from 65.00 to 55.60, down 9.4 points; task expression fell from 43.80 to 37.50, down 6.3 points. The integrity rating remained at pass.
Cause Analysis
The material constraint dimension fell 19.3 points in a single day, far exceeding the 5-point drop in code execution, indicating that this fluctuation stems mainly from how faithfully the model handles source material. The Smoke evaluation uses only 10 questions per day, 2 per dimension; if the randomly drawn questions include cases requiring strict adherence to given materials, deviations in model output will directly drag down that dimension's score. Engineering judgment and task expression declined in tandem, showing that the model also deviated more on questions requiring engineering trade-offs and task phrasing.
Current data cannot distinguish between random question-draw fluctuation and genuine model degradation. The sharp drop in material constraint is more likely caused by a higher proportion of high-difficulty constraint questions in a single draw rather than an overall decline in model capability, since code execution remains at a high level of 94.50.
Implications for Users
Teams that rely heavily on code execution can continue using Claude Opus 4.7, as its code execution score remains high. Scenarios that depend on fidelity to source material—such as contract review, policy interpretation, and question answering over long documents—require an additional manual review step, because the material constraint score has fallen to 70.00.
For developers, when calling Claude Opus 4.7 on tasks that require strict citation of source material, it is advisable to add stronger constraint instructions to the prompt or to call other models in parallel to cross-validate outputs.
Strategic Assessment
Today's main leaderboard score of 83.48 is below yesterday's 94.91, mainly dragged down by material constraint. The stability dimension score of 31.7 suggests that the model's scores on similar questions fluctuate considerably and that consistency is insufficient. The next Smoke evaluation should continue to monitor whether material constraint recovers, in order to determine whether this was an incidental fluctuation.
If material constraint remains low in the next round, Claude Opus 4.7's applicability in material-sensitive scenarios will decline further. The conclusion supported by current data is that the material constraint dimension has become its most notable weakness.
Data source: YZ Index | Run #426 | View raw data
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接