GPT-6 Luna Material Constraints Score Drops 15.9 Points While Code Execution Rises 19.7 Points

In today's Smoke evaluation, GPT-6 Luna's material constraints score fell from 87.90 to 72.00 points, a drop of 15.9 points, while code execution rose from 74.30 to 94.00 points, an increase of 19.7 points. Its main leaderboard score rose from 80.40 to 84.10 points.

Score Comparison and Fluctuation Characteristics

Code execution was 74.30 points yesterday and 94.00 points today; material constraints were 87.90 points yesterday and 72.00 points today; engineering judgment was 100.00 points yesterday and 75.00 points today; task expression was 66.30 points yesterday and 100.00 points today. The main leaderboard score rose by 3.7 points, and the integrity rating remained pass.

Possible Causes

The Smoke evaluation contains only 10 questions per day, with 2 per dimension, so sampling fluctuation is the main explanation. Material constraints and engineering judgment both fell while task expression rose sharply, indicating that changes in the distribution of question types triggered different capabilities. The large increase in code execution also supports this mechanism: the new questions may better suit the model's code-generation path, while material-constraint questions may include stricter citation or formatting requirements.

The likelihood of genuine degradation is low. The rise in the main leaderboard score shows no systemic decline in overall capability. If a model degrades, multiple core dimensions usually weaken in tandem, whereas here gains in code execution and task expression offset the loss in material constraints.

Implications for Users

Teams that rely heavily on material constraints, such as those working on legal contract extraction, academic citation review, or product specification comparison, need to add manual review steps when calling GPT-6 Luna. The improved code execution performance has a positive impact on automated script generation and data-processing pipelines, and such developers can prioritize testing the new version.

Engineering judgment fell to 75.00 points, suggesting that in scenarios requiring multi-step decisions or architecture choices, model output consistency may decrease. Task expression rose to 100.00 points, indicating excellent instruction-following performance on the current question set and suitability for rapid prototyping or conversational tasks.

Strategic Judgment

Based on the current score comparison, this drop in material constraints is more likely a single-day question sampling fluctuation rather than genuine model degradation. The fact that the main leaderboard score rose by 3.7 points supports this judgment. For the next Smoke evaluation, it is recommended to focus on whether material constraints recover and on the magnitude of fluctuations in engineering judgment and task expression. If material constraints remain below 75 points for two consecutive periods, then consider adjusting selection priorities.

Current data do not justify downgrading GPT-6 Luna's overall rating. Developers can continue to use it, but scenarios sensitive to material faithfulness should retain a manual review process.

Data source: YZ Index | Run #404 | View raw data