Claude Opus 4.7 Main Leaderboard Plummets 20.1 Points, Code Execution Falls 28.1 Points in a Single Day

Claude Opus 4.7 in today's Smoke evaluation fell on the main leaderboard from 93.54 to 73.45; the Code Execution dimension dropped from 97.00 to 68.90, and Material Constraints fell from 89.30 to 79.00.

Data Fact Breakdown

The Code Execution dimension fell 28.1 points in a single day, and Material Constraints fell 10.3 points; together they dragged the main leaderboard down by 20.1 points. Engineering Judgment rose from 88.90 to 100.00, and Task Expression rose from 45.60 to 63.90. Integrity Rating remained pass.

The Smoke evaluation includes only 2 questions per dimension per day, so the sample size is extremely small. The 28.1-point drop in Code Execution suggests that the 2 code questions drawn that day may have concentrated on exposing execution failures. The 10.3-point decline in Material Constraints was smaller, indicating that performance on constraint-type questions was relatively stable.

Cause Analysis: Draw Fluctuation or Real Degradation

Code Execution and Material Constraints declined at the same time, but the side leaderboard showed clear gains in Engineering Judgment and Task Expression, indicating that the model's overall output structure did not suffer a systemic collapse. The largest drop was concentrated in Code Execution, pointing to the possibility that the day's questions involved complex multi-step reasoning or strict formatting requirements, and Claude Opus 4.7's execution path broke down on those specific questions.

Material Constraints fell 10.3 points, a smaller decline than Code Execution, suggesting that the model still has a certain degree of stability in citation and faithfulness control. Both side-leaderboard dimensions rose, reflecting better performance that day in judgment logic and clarity of expression, further reducing the likelihood of "overall degradation."

Overall, question-difficulty fluctuation caused by the single-day draw of 10 questions is the most direct mechanism explaining this main-leaderboard plunge. Genuine capability degradation usually comes with simultaneous declines across multiple dimensions, whereas this instance combines rising side-leaderboard scores with falling main-leaderboard scores, pointing to draw factors as dominant.

Specific Implications for Users

For teams that rely heavily on Code Execution, today's data suggests adding a manual review step for critical tasks. A Code Execution score of 68.90 means that the questions drawn that day may have triggered execution errors or format mismatches, so a fallback plan should be prepared when calling the model in production environments.

For scenarios sensitive to Material Constraints, such as document generation and citation verification, 79.00 is still within the usable range, but it is lower than yesterday's 89.30; users can mitigate the impact by reinforcing citation requirements through prompts.

Rising Engineering Judgment and Task Expression, by contrast, provide a positive signal for scenarios requiring logical judgment and clear output. Developers may consider retaining Claude Opus 4.7 in mixed tasks while splitting out the Code Execution portion to other models.

Strategic Judgment

This main-leaderboard plunge was driven mainly by sharp volatility in the single Code Execution dimension, while Material Constraints declined only modestly and the side leaderboard actually improved, indicating that the model has not suffered systemic capability degradation. The single-day Smoke evaluation sample size is too small, and a 28.1-point drop is statistically within the normal range.

If Code Execution remains below 70 in the next Smoke evaluation, multi-day continuous tracking should be initiated to confirm whether it is a genuine degradation signal. The current single-day data is insufficient to determine whether the model is overestimated or underestimated.

It is advisable to monitor the next period's simultaneous performance in Code Execution and Material Constraints as a basis for deciding whether to adjust model selection strategy. If the side leaderboard continues to rise, it can be regarded as compensatory performance in non-main-leaderboard capabilities.

Data from: YZ Index | Run #381 | View raw data