In today's Smoke evaluation, GPT-5.5's code execution score fell from 100.00 yesterday to 75.00, and its main leaderboard score dropped from 78.49 to 73.20, declines of 25 points and 5.3 points, respectively.
Breaking Down the Data
Comparing yesterday's and today's scores, code execution fell 25 points, material constraints rose from 52.20 to 71.00, engineering judgment rose from 75.00 to 100.00, task expression fell from 100.00 to 91.70, and the integrity rating remained pass. The main leaderboard is weighted from code execution and material constraints; today's 73.20 directly reflects the sharp drop in code execution being partially offset by material constraints.
Cause Analysis: Question-Sampling Fluctuation or Genuine Degradation
The Smoke evaluation has only 10 questions per day, 2 per dimension, so single-day fluctuation is within the normal range. While code execution plunged, material constraints and engineering judgment rose significantly, indicating no systematic degradation in the model's overall output capability. The most likely cause is that today's code execution questions, by difficulty or type sampling, skewed toward complex scenarios, causing a concentrated score decline. The small 8.3-point drop in task expression is also consistent with small-sample randomness rather than changes at the model parameter level.
The combination of a 75.00 code execution score and a 71.00 material constraints score caused the main leaderboard to fall only 5.3 points, demonstrating a hedging effect across dimensions.
Specific Implications for Users
Development teams that rely heavily on code execution need to additionally verify code snippets generated by GPT-5.5 today, especially in scenarios involving multi-step logic or edge cases. The higher material constraints score means that document-generation tasks with high requirements for long-text faithfulness and instruction following are actually more stable. When enterprises select models, they should monitor the code execution dimension separately rather than looking only at the main leaderboard score of 73.20.
Strategic Judgment
Based on the single-day comparison, GPT-5.5's code execution decline is more likely fluctuation caused by question sampling rather than genuine model degradation. Engineering judgment rising to 100.00 further supports this judgment. It is recommended that the next Smoke evaluation focus on whether code execution rebounds above 90; if it remains below 80 for two consecutive days, then consider changes in model capability. Current data does not support a conclusion that GPT-5.5's overall capability has been downgraded.
The gap between today's main leaderboard score of 73.20 and yesterday's 78.49 is mainly driven by the single dimension of code execution; the 18.8-point increase in material constraints has partially hedged the impact. Developers can continue to use GPT-5.5 for mixed tasks in production environments, but should add manual review steps to pure code-generation workflows.
The integrity rating remained pass, indicating no systematic hallucinations or policy-violating outputs in the model's responses. No data was provided for the stability dimension this time, so consistency cannot be directly assessed, but the 25-point single-day fluctuation already suggests that multiple days of samples should be accumulated before making a judgment.
Taking all comparison values together, GPT-5.5's performance today is an extreme value within the normal sampling range and does not constitute a signal that deployment strategy needs immediate adjustment. If code execution remains below 80 in subsequent evaluations, that will become an anomaly worth verifying.
Data source: YZ Index | Run #399 | View raw data
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接