GPT-o3 Tops with 94.51 Points: 2026-10-17 Smoke Quick Test Data Brief

The 2026-10-17 YZ Index Smoke quick test covered 14 models, with GPT-o3 ranking first for the day at 94.51 points. Smoke is a daily 10-question quick test, suitable for observing short-term signals, and is not equivalent to Full weekly leaderboard conclusions.

This Smoke evaluation covers only two main leaderboard dimensions: code execution and material constraints. The main leaderboard formula is 0.55 × code execution + 0.45 × material constraints. Because the daily sample size is small, single-day scores are better used as monitoring signals rather than long-term conclusions about model capabilities.

Daily Rankings

Data Interpretation

In today's YZ Index Smoke quick test, the leading models showed clear divergence in how code execution and material constraints combined. GPT-o3 scored 100 in code execution and 87.8 in material constraints for a main leaderboard score of 94.51; Grok 4 achieved 88.69 on the main leaderboard through a balanced combination of 91.7 in code execution and 85 in material constraints; Gemini 3.1 Pro and GPT-6 Sol both scored 100 in code execution but 72.8 and 71.8 in material constraints, respectively, placing third and fourth. Claude Opus 4.7 stood out in the opposite direction, with 95 in material constraints but 75 in code execution, for a main leaderboard score of 84. DeepSeek V4 Pro also scored 100 in code execution and 70 in material constraints, for a main leaderboard score of 86.5. These patterns show that if high-code-execution models have insufficient material constraints, their main leaderboard ranking is constrained by the 0.45 weight.

Among models with notable changes, GLM-4.6 rose 35.7 points on the main leaderboard, with code execution up 16.7 points and material constraints up 58.9 points, while integrity changed from pass to warn; GPT-6 Sol rose 34.3 points on the main leaderboard, with code execution up 55.2 points; GPT-o3 rose 33.8 points on the main leaderboard, with code execution up 61.7 points; Gemini 3.1 Pro rose 18.1 points on the main leaderboard, with code execution up 45.5 points but material constraints down 15.3 points. In terms of anomalous signals, Gemini 3.1 Pro's material constraints plunged 15.3 points, DeepSeek V4 Pro plunged 18.1 points, GPT-6 Astra plunged 16.3 points, Gemini 2.5 Pro plunged 22.6 points, and Doubao Pro's main leaderboard score plunged 16.1 points. These may all stem from single-day small-sample sampling fluctuations or temporary performance degradation, and require confirmation through follow-up runs using the same methodology.

Overall, as a small-sample single-day signal, the Smoke quick test only reflects the immediate combination of code execution and material constraints for the 14 models that day; it does not provide a judgment on long-term stability. Qwen3 Max did not participate in the rankings due to incomplete data caused by an API failure.

Major Changes

• GLM-4.6: main leaderboard up 35.7 points, code execution +16.7 points, material constraints +58.9 points, integrity pass→warn
• GPT-6 Sol: main leaderboard up 34.3 points, code execution +55.2 points, material constraints +8.7 points
• GPT-o3: main leaderboard up 33.8 points, code execution +61.7 points
• Gemini 3.1 Pro: main leaderboard up 18.1 points, code execution +45.5 points, material constraints -15.3 points
• GPT-5.5: main leaderboard up 17 points, code execution +28 points

Signals to Watch

• Gemini 3.1 Pro: material constraints plunged -15.3 points
• DeepSeek V4 Pro: material constraints plunged -18.1 points
• GPT-6 Astra: material constraints plunged -16.3 points
• Gemini 2.5 Pro: material constraints plunged -22.6 points
• Doubao Pro: main leaderboard plunged -16.1 points
• Qwen3 Max: incomplete data (missing several dimensions, API failure/timeout); an automatic rerun has been initiated, and it does not participate in the rankings for this period

When reading this kind of Smoke brief, the focus should be on two questions: first, whether a model has exposed the same type of weakness for multiple consecutive days; second, whether its integrity rating has moved from pass to warn or fail. Large single-day changes in execution or constraint scores may come from question sampling, or they may be early signals of genuine degradation and require follow-up run confirmation.

Data source: YZ Index | Run #382 | View raw data