Claude Opus 4.7's main benchmark score in today's Smoke evaluation dropped from 100.00 to 80.09, with the code execution dimension falling from 100.00 to 75.00, and the material constraint dimension dropping from 100.00 to 86.30.
Decomposition of Main Benchmark Score Changes
The main benchmark is composed solely of the weighted scores of code execution and material constraint. Code execution fell by 25 points in a single day, directly dragging down the main benchmark by 19.9 points. Material constraint dropped by 13.7 points, contributing the remaining decline. Both dimensions achieved perfect scores yesterday but showed significant losses today.
Simultaneous Decline in Sub-indicators
Engineering judgment fell from 100.00 to 55.60, a drop of 44.4 points. Task expression dropped from 75.80 to 58.90, a decline of 16.9 points. The integrity rating remained at "pass," not triggering the threshold.
Possible Cause Analysis
The Smoke evaluation features only 10 questions per day, with 2 questions per dimension, and the question selection is highly random. Given that both code execution and material constraint achieved perfect scores yesterday but lost points today simultaneously, the most likely reason is that the randomly selected questions today were more difficult, rather than a change in model parameters. The largest drop in the engineering judgment sub-indicator is also consistent with the characteristic that score losses in individual questions are amplified in a small sample.
If the model were genuinely degrading, we would typically observe systematic losses across multiple dimensions over consecutive days. Currently, with only single-day data, it is impossible to distinguish between random sampling fluctuations and actual capability degradation.
Implications for Users
Development teams heavily reliant on code execution should add manual review steps for critical tasks. For RAG scenarios sensitive to material constraint, today's score of 86.30 remains within the usable range, though lower than yesterday's perfect performance.
The engineering judgment sub-indicator of 55.60 suggests that in scenarios requiring complex decision chains, the consistency of model output may decrease, and it is recommended to increase multi-round verification steps.
Strategic Assessment
Based on the single-day comparison, Claude Opus 4.7's main benchmark score of 80.09 still exceeds the baseline of most competitors. However, the 25-point drop in code execution requires continuous tracking in the same dimension over the next period to confirm whether it is merely a random fluctuation. The 44.4-point drop in engineering judgment is the most critical signal to verify in this evaluation.
Data Source: YZ Index | Run #234 | View Raw Data
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接