In today's Smoke evaluation, GPT-6 Luna's main leaderboard score fell from 87.27 to 75.79, a drop of 11.5 points. The core driver was the code execution dimension, which plunged from 100.00 to 74.30, a decline of 25.7 points.
Breaking Down the Data: Divergence Between Main and Side Leaderboards
The main leaderboard is composed of only two dimensions: code execution and material constraints. Today code execution scored 74.30 and material constraints 77.60, yielding a weighted score of 75.79. Yesterday the same model scored 100.00 on code execution and 71.70 on material constraints, for a main leaderboard score of 87.27. Material constraints actually rose 5.9 points but failed to offset the steep slide in code execution.
Engineering judgment rose from 50.00 to 95.50, and task expression rose from 67.50 to 83.80. Both side-leaderboard dimensions improved substantially, moving in the exact opposite direction of the main leaderboard.
Possible Causes: Sampling Noise or Genuine Degradation
The Smoke evaluation uses only 10 questions per day, 2 per dimension. Single-day score fluctuations fall within the normal range. The 25.7-point drop in code execution this time exceeds the typical random sampling interval, suggesting a possible decline in the model's answer consistency.
The integrity rating moved from pass to warn, the only newly triggered threshold signal in this evaluation. This change coincided with the large loss in the code execution dimension, pointing to possible output instability or deviation from constraints in code-related tasks.
The loss in the code execution dimension was the sole decisive factor in the main leaderboard decline.
What This Means for Users
Teams that rely heavily on code execution should immediately lower their trust level in GPT-6 Luna. A score of 74.30 means that in scenarios requiring precise code generation or debugging, the model's output accuracy has visibly declined.
The material constraints dimension holds at 77.60, so scenarios with high demands on content fidelity can still use the model, but manual verification steps should be added. The high side-leaderboard scores in engineering judgment and task expression are only of reference value in non-core workflows.
Strategic Assessment
This 11.5-point main leaderboard decline was driven primarily by a single dimension, code execution, and was accompanied by an integrity rating downgrade — a signal that warrants close attention. Large gains on the side leaderboards cannot offset the decline in the main leaderboard's core capability.
The next evaluation should focus on whether code execution can return to the 90-plus range and whether the integrity rating recovers to pass. If code execution stays below 80 for two consecutive periods, the model can be judged to have systematically degraded in that dimension.
Current data support only the conclusion that "a single-day anomaly requires verification," not the judgment that "the model has failed overall." We recommend temporarily removing GPT-6 Luna from code-intensive scenarios and keeping it in light-code tasks dominated by material constraints for continued observation.
Data source: YZ Index | Run #410 | View raw data
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接