GPT-5.5 Falls to 54.16 by Week's End, Down 38.3; Doubao Pro Posts 71.7 Mean with 52.2 Volatility

According to Smoke evaluation data from 2026-11-16 to 2026-11-22, GPT-5.5 fell from 92.44 points on the first day to 54.16 points on the final day, a trend decline of 38.3 points, with a mean of 73.4 points and volatility of 38.3 points.

Key Facts on the Declining Models

GPT-5.5: first day 92.44, final day 54.16, trend -38.3, mean 73.4, volatility 38.3; GPT-6 Sol: first day 92.44, final day 64.63, trend -27.8, mean 76.6, volatility 27.8; GPT-6 Luna: first day 88.75, final day 67.13, trend -21.6, mean 74.6, volatility 32.8; GPT-6 Astra: first day 78.69, final day 66.79, trend -11.9, mean 75.1, volatility 26.3; GPT-6.1 Sol: first day 78.69, final day 70.88, trend -7.8, mean 75.2, volatility 25.3; Claude Sonnet 4.6: first day 70.74, final day 61.56, trend -9.2, mean 69.3, volatility 15.9; GLM-4.6: first day 51.19, final day 43.77, trend -7.4, mean 53.1, volatility 28.2.

Comparison of Rising Models and Volatility

Rising models include Grok 4: first day 81.99, final day 83.51, trend +1.5, mean 72, volatility 34.8; GPT-o3: first day 68.46, final day 78.4, trend +9.9, mean 71.2, volatility 20.5; Gemini 3.1 Pro: first day 53.69, final day 57.52, trend +3.8, mean 60.3, volatility 19.1; DeepSeek V4 Pro: first day 51.19, final day 62.33, trend +11.1, mean 58.3, volatility 42.2; Gemini 2.5 Pro: first day 42.69, final day 57.52, trend +14.8, mean 59, volatility 44.8; Doubao Pro: first day 40.82, final day 73.17, trend +32.4, mean 71.7, volatility 52.2. Claude Opus 4.7: first day 82.43, final day 78.09, trend -4.3, mean 79.1, volatility 19.8. Qwen3 Max: first day 0, final day 0, trend 0, mean 0, volatility 0.

Causes: Execution and Material Constraint Dimensions

The Smoke evaluation focuses on two main leaderboard dimensions: code execution and material constraints. Models such as GPT-5.5 and GPT-6 Sol scored high on the first day before declining continuously, most likely due to a drop in consistency on code execution tasks; fluctuations in material constraint scores directly dragged down the overall trend. Doubao Pro and DeepSeek V4 Pro trended upward, but their volatility reached 52.2 and 42.2 points respectively, indicating that their code execution outputs differed markedly across dates and that material constraints may at times be strict and at other times loose. GLM-4.6's integrity rating shifted from pass to warn, indicating signals of inconsistent material citation in its responses, which directly affects the credibility of the engineering-judgment side of the leaderboard.

Implications for Users

Teams that rely heavily on code execution should note that GPT-5.5 has a mean of 73.4 points but volatility of 38.3 points; score swings within a single seven-day period may cause unstable output from automated scripts. In scenarios sensitive to material fidelity, after GLM-4.6's integrity rating turned to warn, developers should add manual verification steps to prevent citation errors from accumulating. Doubao Pro's mean of 71.7 points is close to GPT-6 Sol's, but volatility of 52.2 points means developers relying on it to generate long code need a multi-round retry mechanism.

Strategic Assessment

GPT-5.5 and GPT-6 Sol have means of 73.4 and 76.6 points respectively, yet show the largest trend declines, a signal that their execution consistency under the current Smoke rapid test is already lower than that of Grok 4 and GPT-o3. DeepSeek V4 Pro and Doubao Pro trend positive, but their high volatility values suggest lower stability-dimension scores, making them suitable for short-term validation rather than long-term production dependence. The change in GLM-4.6's integrity rating is the only access-threshold signal this week, and its pass/warn status should continue to be tracked next period to judge whether it recovers.

Claude Opus 4.7 has a mean of 79.1 points and volatility of 19.8 points, the smallest decline among the falling models, indicating that its material constraint dimension is relatively robust. Gemini 2.5 Pro and Doubao Pro ended the final day at 57.52 and 73.17 points respectively, but the former's volatility is 44.8 points and the latter's 52.2 points, indicating that upward trends come with high inconsistency; users should first verify the standard deviation of per-question code execution scores.

This week's data covers only a seven-day, ten-question rapid test. Qwen3 Max's continued 0 points shows that it produced no valid execution output in the Smoke evaluation. All judgments derive from first-day versus final-day comparisons and from mean and volatility figures, with no external versions or industry events introduced.


Data source: YZ Index | Run #438 | View raw data

This article is from Winzheng Index blog, translated in full by Winzheng (winzheng.com). Click here to view the original When republishing the translation, please credit the source. Thank you!