GPT-6 Astra Material Constraints Plunge 19 Points While Code Execution Soars 25, Main Leaderboard Defies Trend with 5.2 Gain

In today's Smoke evaluation, GPT-6 Astra's material constraint score fell from 89.30 yesterday to 70.30, a drop of 19 points; its code execution score rose from 72.00 to 97.00, an increase of 25 points; its main leaderboard score rose from 79.79 to 84.99, a net gain of 5.2 points. Engineering judgment rose from 75.00 to 100.00, task expression remained unchanged at 79.70, and the integrity rating remained at pass.

Breaking Down the Data

The material constraint dimension lost 19 points in a single day, directly lowering its weighted contribution to the main leaderboard. The code execution dimension gained 25 points over the same period, offsetting the material constraint loss and driving a net 5.2-point rise on the main leaderboard. The two core auditable dimensions moved in completely opposite directions, producing a single-day spread of 44 points.

The Smoke evaluation has only 2 questions per dimension per day, 10 questions in total. If the two material constraint questions each lost about 9.5 points, that would cause a 19-point overall decline; if the two code execution questions each gained 12.5 points, that would cause a 25-point overall rise. In daily small-sample testing, this magnitude falls within the statistically allowable range.

Cause Analysis

Question sampling variance is the most direct explanation. Material constraint questions may involve stricter citation requirements or longer context constraints; on one of the questions, the model clearly deviated from the original text or added unauthorized content, concentrating the point deduction in a single question. Code execution questions may have shifted toward simpler algorithm or data processing tasks, allowing the model to pass all test points at once and score full marks or near full marks.

The possibility of genuine model degradation is relatively low. Material constraints and code execution are both auditable dimensions on the main leaderboard; if the model's underlying capabilities suffered a systemic decline, the two dimensions would usually move in the same direction. The current pattern of one rising and one falling is more consistent with random sampling of different questions that draw on different model abilities, rather than a structural change in the abilities themselves.

Implications for Users

Teams that rely heavily on material constraints—such as those reviewing legal contracts, checking academic citations, or comparing financial statements—should increase the proportion of human review after today. At a material constraint score of 70.30, GPT-6 Astra is noticeably more likely to add unauthorized content in a single response.

Developers in code execution-intensive scenarios—such as writing data-cleaning scripts, validating simple algorithms, or generating automated reports—can continue using the model. Its current score of 97.00 is already near the full-score range, indicating high execution accuracy.

Strategic Assessment

The fact that the main leaderboard score rose 5.2 points shows that the 19-point loss in material constraints did not change GPT-6 Astra's overall ranking position in today's Smoke evaluation. Whether it requires continued attention depends on whether material constraints remain below 80 in the next Smoke evaluation. If material constraints recover to above 85 in the next round, the 19-point drop is more likely to be judged a single-day sampling anomaly; if it remains below 75, material constraints should be considered a key tracking dimension.

Based on the existing score comparison, GPT-6 Astra's performance on the material constraint dimension has shown an obvious single-day anomaly. Users who rely on this dimension are advised to hold off on making GPT-6 Astra their preferred model for material constraints until the next full evaluation results are released.

Data source: YZ Index | Run #373 | View raw data