In today's Smoke evaluation, GPT-6 Luna's code execution score dropped from 99.10 yesterday to 75.00, its material constraint score rose from 62.80 to 94.80, and its main leaderboard score climbed from 82.77 to 83.91.
The Facts Behind the Score Changes
The code execution dimension stood at 99.10 yesterday and 75.00 today, a single-day decline of 24.1 points. The material constraint dimension stood at 62.80 yesterday and 94.80 today, a single-day rise of 32 points. Engineering judgment (secondary leaderboard, AI-assisted assessment) fell from 100.00 to 63.90, and task expression (secondary leaderboard, AI-assisted assessment) fell from 100.00 to 83.80. The main leaderboard score was 82.77 yesterday and 83.91 today, up 1.1 points. The integrity rating remains at pass.
Analysis of Possible Causes
The Smoke evaluation contains only 10 questions per day, 2 per dimension, so the small sample size means question-draw volatility is the main mechanism. The large changes in opposite directions for code execution and material constraint indicate that the code execution questions drawn that day differed significantly in difficulty or type from the previous day, while the material constraint questions were relatively easy. Engineering judgment and task expression declined at the same time, but both are secondary leaderboard dimensions and do not directly affect the main leaderboard calculation. The main leaderboard score rose only slightly, indicating that after the rises and falls of the two core auditable dimensions offset each other, the overall result remained stable.
There is no multi-day consecutive data to support a conclusion of genuine model degradation; a single-day opposing change is more consistent with random sampling characteristics than with a systematic decline in model capability.
Implications for Users
Teams that rely heavily on code execution may encounter a higher failure rate in that day's tasks and should prepare backup plans or manual review. Scenarios sensitive to material constraints, such as document generation and citation checking, may see better performance. Enterprises that select models based on main leaderboard rankings can continue to observe, since the main leaderboard score has not shown any substantial decline.
Strategic Judgment
Based on the existing single-day data, GPT-6 Luna's main leaderboard score remains stable, and the sharp drop in code execution is more likely a result of the question draw than a signal of model degradation. The next evaluation round needs to verify whether code execution rebounds above 90 points; if it stays below 80 for two consecutive days, that would warrant further attention. The large fluctuation in the engineering judgment secondary leaderboard does not change the main leaderboard conclusion.
Overall, the current data does not support a downward revision of the long-term assessment of GPT-6 Luna's code execution capability, and continued tracking of dimension consistency in subsequent Smoke evaluations is recommended.
Data source: YZ Index | Run #432 | View raw data
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接