GPT-5.5 Material Constraints Plunge 18 Points While Code Execution Surges 30.3; Main Leaderboard Rises 8.6

In today's Smoke evaluation, GPT-5.5's Code Execution score rose from 69.50 yesterday to 99.80, while Material Constraints fell from 79.30 yesterday to 61.30; the main leaderboard score rose from 73.91 to 82.48.

Score Comparison and Key Findings

The Code Execution dimension reached 99.80 today, up 30.3 points from yesterday; the Material Constraints dimension was only 61.30 today, down 18 points from yesterday. Engineering Judgment and Task Expression remained unchanged at 75.00 and 90.00, and the Integrity Rating remained pass. The main leaderboard is a weighted composite of the Code Execution and Material Constraints dimensions; today's rise in the main leaderboard to 82.48 mainly came from the large increase in Code Execution.

Possible Causes

The Smoke evaluation has only 10 questions per day, 2 per dimension; the small sample size makes single-day score fluctuations normal. The simultaneous appearance of -18 points in Material Constraints and +30.3 points in Code Execution points to random item sampling: the Code Execution questions selected today may better match the model's current training distribution, while the Material Constraints questions may include more constraints requiring strict fidelity to the provided material. The two dimensions moved sharply in opposite directions, and there is no evidence of systematic degradation in the same direction; therefore, this is more likely sampling fluctuation than a real decline in model capability.

Code Execution at 99.80 and Material Constraints at 61.30 occurring on the same day shows that the model's performance differences across task types have been amplified.

Implications for Users

Teams focused on code execution can have higher confidence under today's data; the 99.80 score indicates that the model performs stably in code generation and debugging scenarios. Scenarios that depend on material fidelity face risk; 61.30 means the model is more likely to deviate from the given material or instructions, making it suitable for auxiliary writing tasks with low requirements for fidelity to the original text. Engineering Judgment and Task Expression remained unchanged, indicating no obvious fluctuation in the model's side-leaderboard capabilities.

Strategic Judgment

Based on today's score comparison, the -18 points in Material Constraints is more likely caused by item sampling fluctuation rather than model degradation. The rise of the main leaderboard to 82.48 shows that Code Execution carries greater weight in the current evaluation. It is recommended to continue tracking the Material Constraints dimension in the next period; only if it stays below 65 for two consecutive days should a real change in the model be considered. Current data do not support a judgment that GPT-5.5's overall capability has been downgraded.

The zero change in Engineering Judgment at 75.00 and Task Expression at 90.00 further confirms that main-leaderboard fluctuations are concentrated in the two auditable dimensions of Code Execution and Material Constraints. Users can allocate weights according to their scenarios: projects with a high share of code execution can prioritize it, while projects sensitive to material constraints need additional manual review steps.

Today's data fully record the dimensional score changes of GPT-5.5 in the Smoke evaluation; Code Execution at 99.80, Material Constraints at 61.30, and the main leaderboard at 82.48 are all direct measurement results. Any inference beyond these values lacks support.

Data source: YZ Index | Run #415 | View raw data