GLM-4.6 Material Constraint Plunges 25 Points, Code Execution Rises 50 Points, Smoke Test Leaderboard Reverses Upward

GLM-4.6 in today's Smoke evaluation saw its material constraint score drop from 50.00 points yesterday to 25.00 points, a decline of 25 points, while its code execution score rose from 25.00 points to 75.00 points, and the leaderboard score increased from 36.25 points to 52.50 points.

Score Comparison and Volatility Magnitude

The specific changes from yesterday to today are: code execution +50 points, material constraint -25 points, engineering judgment dropping from 94.50 points to 75.00 points, and task expression rising from 50.00 points to 75.00 points. The leaderboard overall rose by 16.3 points, with the integrity rating remaining at warn.

Question Sampling Fluctuation or Genuine Model Degradation

The Smoke evaluation consists of only 10 questions daily, with 2 questions per dimension, leading to inherently large single-day standard deviations. The opposite 50-point-level jumps in material constraint and code execution most likely stem from differences in question sampling. Material constraint questions may have drawn cases requiring strict citation of the original text or format constraints, while code execution questions drew relatively simple algorithm implementation tasks. The engineering judgment drop of 19.5 points simultaneously suggests that the day's questions may have trended across multiple dimensions toward directions requiring rigorous reasoning or material fidelity.

If this were genuine model degradation, it would typically manifest as systematic declines in the same dimension over consecutive days, rather than extreme reverse fluctuations on a single day. Currently, with only one day of data, degradation cannot be confirmed.

Implications for Users

In scenarios heavily reliant on material constraint, such as legal contract extraction, academic citation verification, and product specification alignment, GLM-4.6's score of 25.00 points today implies a higher risk of omissions or rewrites of the original text in single responses. Developers in these scenarios need to add manual verification steps or switch to models with higher material constraint scores.

A model with a code execution score of 75.00 points offers improved usability for algorithm problems, data processing script generation, and simple automation tasks. An engineering judgment score of 75.00 points suggests that in system design review and architecture trade-off tasks, the consistency of model outputs may be lower than yesterday's level.

Strategic Assessment

Based on single-day data, the combination of a material constraint score of 25.00 points and an engineering judgment score of 75.00 points is more likely a sampling fluctuation rather than model capability degradation. The leaderboard's rise to 52.50 points is primarily driven by code execution and does not change the signal from the low material constraint score. It is recommended that the next Smoke evaluation focus on whether the material constraint score rebounds to above 40 points; if it remains around 25 points for two consecutive days, longer-cycle targeted tests should be initiated.

For selection teams, the current data does not support recommending GLM-4.6 as the preferred model for material-fidelity-sensitive tasks. The stability dimension score of 31.7 points already indicates significant score fluctuations on similar questions, and the single-day material constraint plunge further confirms this.

Data Source: YZ Index (YZ Index) | Run #226 | View Raw Data