GLM-4.6 Smoke Evaluation Scores 19.50 in Code Execution; Material Constraints Missing, Not Ranked This Period

GLM-4.6 showed incomplete data in today's Smoke Evaluation. Two evaluation dimensions were missing, and an API failure or timeout has placed the model into an automatic rerun process, so it will not participate in the main leaderboard ranking this period.

Score Comparison Data

From yesterday to today, the code execution dimension went from no data to 19.50; the material constraints dimension remains missing, the engineering judgment dimension is missing, the task expression dimension reached 50.00, and the main leaderboard has no overall score.

Data Facts Summary

Smoke Evaluation has only 10 questions per day, with 2 questions corresponding to one dimension; single-day fluctuations are within the normal range. This time, GLM-4.6 has values only for the code execution and task expression dimensions; the remaining dimensions could not be obtained due to API issues, making the main leaderboard calculation impossible.

Cause Analysis

Question draw fluctuations may cause some dimension scores to be low, but the complete absence of the judgment dimension and one other dimension this time is more likely due to API call timeouts or abnormal server-side responses rather than a degradation of the model's own capabilities. The task expression dimension scored 50.00, showing that the model still has output capability on that side of the leaderboard.

Implications for Users

Teams that emphasize code execution should note the low 19.50 score this time when relying on GLM-4.6, as it may affect stability assessments for code generation applications. For scenarios sensitive to material fidelity, because a key dimension is missing, this data cannot provide reference value.

Developers planning to integrate GLM-4.6 into production environments should wait for the complete dimension results after the automatic rerun, and avoid making selection decisions based only on a single dimension score of 19.50 or 50.00.

Strategic Assessment

Based on the available score comparison, GLM-4.6's anomaly this time is mainly caused by API failures rather than genuine model degradation; it is worth verifying in the next period whether the complete scores recover after the rerun. The current data is insufficient to determine its true ranking position on the main leaderboard.

Single-day fluctuations in Smoke Evaluation are normal, but the situation in which missing dimensions cause absence from the main leaderboard needs continued tracking, in order to distinguish incidental API issues from potential capability changes.

Enterprises making selection decisions should prioritize the multi-dimensional data after the rerun; the 19.50 and 50.00 scores should be treated only as records of this anomaly and should not be used directly for long-term capability assessment.

Data source: YZ Index | Run #378 | View raw data