In today's Smoke evaluation, GPT-6 Astra's code execution score fell from yesterday's 97.30 to 70.00, the main leaderboard dropped from 84.70 to 78.69, while material constraints rose from 69.30 to 89.30.
Score Comparison Facts
The specific changes from yesterday to today are: code execution 97.30→70.00, material constraints 69.30→89.30, engineering judgment 100.00→97.80, task expression 94.70→85.00, main leaderboard 84.70→78.69, with the integrity rating remaining at pass.
Cause Analysis
The 27.3-point single-day drop in code execution most likely stems from sampling variance across just two questions. The Smoke evaluation uses only 2 questions per dimension each day, so an uneven difficulty distribution across questions can easily produce large swings. Material constraints rose by 20 points over the same period, indicating improved performance on constraint-following tasks; the opposite movement of these two dimensions does not support a conclusion of systematic degradation. Engineering judgment and task expression declined slightly, each by less than 10 points, further pointing to random question factors rather than an overall capability decline.
Implications for Users
Teams that lean heavily on code execution should apply extra validation to GPT-6 Astra's outputs today, especially in scenarios involving complex computation or multi-step reasoning. The higher material constraints score, by contrast, offers greater confidence for scenarios that require strict adherence to input limits. Developers who rely on main leaderboard rankings should refer to multi-day averages and avoid letting a single-day score of 78.69 directly drive model selection decisions.
Strategic Assessment
Current data shows only an anomalous fluctuation in the code execution dimension and does not yet constitute evidence of genuine degradation. If code execution rebounds above 90 in the next evaluation, it can be judged a sampling event; if it stays below 80, continuous tracking should be initiated. The main leaderboard score of 78.69 remains above the baseline of some competitors, so there is no sufficient reason to lower priority at this time.
The slight declines to 97.80 in engineering judgment and 85.00 in task expression do not change the logic behind how the main leaderboard is composed. The main leaderboard is weighted only by code execution and material constraints, and today's weighted result of 78.69 directly reflects the net change across the two dimensions. An integrity rating of pass indicates that the model showed no refusals to answer or fabricated data.
For organizations selecting models, today's results suggest adding a manual review step in code-intensive projects. The material constraints score of 89.30 can serve as a reference ceiling for this model in compliance scenarios. Overall judgment: this sharp drop is more likely statistical noise from evaluation sampling than a structural decline in model capability.
Data source: YZ Index | Run #429 | View raw data
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接