GLM-4.6 Smoke Test Scores 25 on Code Execution; Main Leaderboard 36.25 as API Failure Leaves Communication Dimension Missing

In today's Smoke evaluation, GLM-4.6 scored 36.25 on the main leaderboard, with 25.00 in the code execution dimension, 50.00 in material constraints, and 25.00 in engineering judgment. Because data for the communication dimension was missing, it did not enter the official ranking.

Data facts: dimension scores and what was missing

This Smoke evaluation is a daily 10-question quick test, with 2 questions per dimension. GLM-4.6 scored 25.00 in code execution, 50.00 in material constraints, and 25.00 in engineering judgment, averaging 36.25 on the main leaderboard. The communication dimension was completely missing due to API failure/timeout, and the task expression dimension also had no valid record. Yesterday's comparison showed all dimensions as "-"; today specific values appeared for the first time.

Cause analysis: question-draw variance or API failure

Single-day fluctuation in Smoke evaluations is normal, but the complete absence of the communication dimension this time points directly to an API failure/timeout rather than a simple change in question difficulty. Code execution and engineering judgment both came in at 25.00, while material constraints was relatively higher at 50.00, indicating a break in tasks that require stable API responses. Question-draw variance typically moves scores slightly within the 40-60 range, whereas a low 25.00 accompanied by a missing dimension is more likely to stem from a technical-level connection interruption than from a sudden degradation of the model's capabilities.

GLM-4.6's communication dimension was missing in this Smoke evaluation due to API failure/timeout; its main leaderboard score of 36.25 did not participate in the ranking.

What this means for users

Teams that lean heavily on code execution should build in API retry mechanisms when calling GLM-4.6 for programming tasks, because the 25.00 shows the model has already exhibited unstable responses in the Smoke quick test. For scenarios sensitive to material fidelity, 50.00 remains acceptable, but the 25.00 in engineering judgment suggests that in development workflows requiring multi-step reasoning, the model's output consistency may fall short of expectations. Developers relying on this model should add timeout monitoring in production environments to avoid a single API failure breaking the entire task chain.

Strategic judgment: is continued attention warranted

Based on the current score comparison, GLM-4.6's anomaly this time was mainly caused by an API failure rather than genuine model degradation. The inherent fluctuation of a 10-question single-day quick test is not enough to explain a completely missing dimension, so the 36.25 is not representative. If the communication dimension recovers in the next Smoke evaluation and code execution stays around 25.00, the model should be placed on a key watch list; if it recovers above 50, this incident can be treated as an isolated technical problem and no adjustment to model selection strategy is needed.

When enterprises select models, they should consider GLM-4.6's Smoke main leaderboard score together with the full dimension record; looking at 36.25 alone may underestimate its actual usability. Developers should prioritize testing API stability during integration rather than relying directly on a single day's quick-test score.

Data source: YZ Index | Run #418 | View raw data