Smoke 7-Day Data: Gemini 2.5 Pro Trends +34.3 Points Leading, GLM-4.6 Volatility 40.5 Points Most Severe

According to Smoke evaluation data from 2026-07-08 to 2026-07-12, Gemini 2.5 Pro trended up by 34.3 points over 7 days, rising from 59.9 points on the first day to 94.24 points on the last day, with an average of 75.9 points and a volatility of 34.3 points.

Score Trajectory of Rising Models

Doubao Pro trended up by 20.2 points, from 71.83 points on the first day to 91.99 points on the last day, average 82.8 points, volatility 20.2 points. Claude Opus 4.7 trended up by 15.6 points, from 81.44 points to 96.99 points, average 84.3 points, volatility 25.7 points. Qwen3 Max trended up by 16.4 points, from 77.83 points to 94.24 points, average 79.6 points, volatility 24 points. GPT-5.5 trended up by 14.2 points, from 66.09 points to 80.27 points, average 75.4 points, volatility 14.2 points. Gemini 3.1 Pro trended up by 19.5 points, from 61 points to 80.49 points, average 76.6 points, volatility 28.2 points. Grok 4 trended up by 5.3 points, from 91.69 points to 96.99 points, average 86 points, volatility 22.4 points. Claude Sonnet 4.6 trended up by 10.7 points, from 86.28 points to 96.99 points, average 82.4 points, volatility 31 points. DeepSeek V4 Pro trended up by 1.8 points, from 95.19 points to 96.99 points, average 86 points, volatility 29.3 points. GLM-4.6 trended up by 13.8 points, from 38.75 points to 52.5 points, average 52.7 points, volatility 40.5 points.

Performance of the Only Declining Model

GPT-o3 trended down by 3.2 points, from 83.44 points on the first day to 80.27 points on the last day, average 82.5 points, with a volatility of just 6.6 points. The model's scores remained relatively stable over 7 consecutive days, without large fluctuations.

Cause Analysis of High-Volatility Models

GLM-4.6 had a volatility of 40.5 points, far higher than other models, with its scores swinging widely in the range of 38.75 to 52.5 points. Gemini 2.5 Pro volatility 34.3 points, Claude Sonnet 4.6 volatility 31 points, DeepSeek V4 Pro volatility 29.3 points. These volatilities stem from differences in the models' consistency in answering similar questions in Smoke's daily 10-question quick tests. The stability dimension formula max(0, 100-stddev×2) directly reflects the standard deviation of scores; GLM-4.6's high standard deviation leads to its low stability score.

Signals of Integrity Rating Changes

GLM-4.6's integrity rating changed from fail to pass and then to warn, with consecutive warn signals. Doubao Pro, GPT-5.5, and Grok 4 all experienced fluctuations from pass to warn and back to pass. As an entry threshold, these warn records directly affect the models' credibility in the material constraint and code execution dimensions.

Specific Implications for Users

Teams with a focus on code execution should prioritize DeepSeek V4 Pro, which has an average of 86 points and a last-day score of 96.99 points, with a volatility of 29.3 points but overall stable position. For scenarios sensitive to material fidelity, avoid GLM-4.6, as its integrity rating warn and 40.5-point volatility may lead to inconsistent outputs. Developers relying on continuous tasks can consider GPT-o3, which has a volatility of only 6.6 points, suitable for scenarios requiring stable outputs.

Strategic Assessment

DeepSeek V4 Pro, with an average of 86 points and a positive trend, is currently undervalued. GLM-4.6, with high volatility and an integrity warn, is currently overvalued. Gemini 2.5 Pro's 34.3-point trend warrants further verification of its consistency source in the next round. GPT-o3's -3.2-point decline is the only negative signal, requiring attention to its subsequent material constraint performance.

7-day data only covers Smoke's daily 10-question sample. Rising models such as Gemini 2.5 Pro and Doubao Pro have approached the 96.99-point ceiling on the last day, indicating improvement in the material constraint dimension. The declining model GPT-o3 still maintains an average of 82.5 points, higher than the average of most rising models, indicating that its baseline capability is not affected by the trend.

Models with volatility exceeding 30 points include GLM-4.6, Gemini 2.5 Pro, and Claude Sonnet 4.6. The potential instability of these models on the engineering judgment benchmark and task expression benchmark requires additional testing during enterprise selection. Models with an integrity rating of warn have already triggered the threshold in the integrity dimension; before actual deployment, separate audits of code execution and grounding scores are needed.

Data source: YZ Index | Run #226 | View raw data