Claude Opus 4.7 Code Execution Plunges 17.4 Points, Task Expression Drops 31.2

In today's Smoke evaluation, Claude Opus 4.7's code execution score fell from 75.00 to 57.60, and its main leaderboard score dropped from 73.61 to 69.53.

Score Comparison Data

Material constraints rose from 71.90 to 84.10, engineering judgment fell from 72.20 to 69.50, task expression dropped from 57.50 to 26.30, and the integrity rating remained at pass. The Smoke evaluation contains only 10 questions per day, so single-day fluctuations fall within the normal range.

Analysis of the Fluctuation

The simultaneous sharp pullback in code execution and task expression most likely stems from a change in the difficulty distribution of the questions drawn that day. The Smoke evaluation includes only 2 questions per dimension, so a single misstep can cause a swing of more than 10 points. The material constraints score moved in the opposite direction, rising 12.2 points, indicating that the model remains stable on constraint-compliance questions and that this is not an overall capability regression.

Engineering judgment declined only slightly, by 2.7 points, further confirming that the main leaderboard drop was driven primarily by the code execution and task expression dimensions rather than by a systematic decline in the model's underlying capabilities.

Implications for Users

Development teams that rely heavily on code execution should note that the single-day Smoke score of 57.60 may reflect performance on the specific question types drawn that day and does not directly equate to long-term coding ability. The rise in the material constraints score to 84.10 shows that Claude Opus 4.7 remains highly reliable in scenarios requiring strict instruction adherence.

The low task expression score of 26.30 may affect tasks that require clearly structured output, and enterprises should consider multi-day data when making selection decisions.

Strategic Assessment

Based on single-day data, Claude Opus 4.7's main leaderboard score of 69.53 represents a 4.1-point decline from yesterday's 73.61, with the core driver being sampling variance in code execution and task expression. The integrity rating remains at pass, and the model's baseline compliance is unaffected.

The next evaluation should focus on verifying whether code execution and task expression recover. If both dimensions remain low, closer attention should be paid to the model's stability on the corresponding task types; if they recover to the range above 75 points, this decline is more likely attributable to question sampling rather than genuine degradation.

Data source: YZ Index | Run #402 | View raw data