In the Smoke quick test for Run#243 on 2026-07-23, GLM-4.6 scored 55.74 on the main leaderboard, 25.00 on code execution, 93.30 on material constraint, and received an integrity rating of fail (probe score 30.00).
Dimensional Contrast Revealed by Score Structure
The 93.30 score on material constraint contrasts sharply with the 25.00 on code execution. Material constraint assesses whether a model can strictly adhere to given long documents and correctly cite sources. GLM-4.6's high score in this dimension indicates good fidelity to provided materials. A code execution score of 25.00 shows a low pass rate in the real Python sandbox, with frequent failures in completing executable code tasks.
The integrity rating of fail (probe 30.00) is an independent signal. This dimension uses 42 canary probes to detect whether a model treats fictitious entities as real references; a score of 30.00 indicates that probes were triggered. The probe score is unrelated to the material constraint score, meaning that even if a model can properly cite given materials, it may still fabricate sources when no materials are provided.
Comparison with Other Models on the Same Day
All 11 models tested on the same day received a pass on integrity. Gemini 2.5 Pro scored 100.00 on probes, Claude Opus 4.7 and GPT-5.5 both scored 90.00, Qwen3 Max, Grok 4, GPT-o3, Claude Sonnet 4.6, and Gemini 3.1 Pro each scored 80.00, Doubao Pro scored 75.00, and DeepSeek V4 Pro scored 65.00. GLM-4.6 was the only model to receive a fail, with a probe score 35 points lower than the next lowest, DeepSeek V4 Pro.
Historical Run Data
In GLM-4.6's public Smoke history, three instances of fail or warn occurred on 2026-07-23, 2026-07-22, and 2026-07-20. On 2026-07-22 Run#241, the main leaderboard score was 74.00 (code execution 97.00, material constraint 75.00, probe 25.00). On 2026-07-21 Run#240, the main score was 62.83 with an integrity pass (probe 80.00). Runs marked as 0 are not used as baselines for comparison.
Causal Analysis
The coexistence of a high material constraint score and an integrity fail indicates that while the model controls citations well when explicit materials are provided, it tends to generate references to fictitious entities in scenarios lacking materials or involving probes. The contrast between a code execution score of 25.00 and a material constraint score of 93.30 may stem from the fact that coding tasks require precise execution rather than text matching; the model is prone to syntax or logic errors when generating runnable code.
Implications for Users
For teams relying on code execution, GLM-4.6's current score of 25.00 means additional manual verification or post-processing is necessary in deployment, increasing development costs. For scenarios sensitive to material fidelity—such as contract review or research report generation—the 93.30 score provides some assurance, but the integrity fail necessitates adding a manual source verification step to avoid citing nonexistent literature or data.
Strategic Assessment
GLM-4.6's integrity fail is a negative signal independent of material constraint. Three fails have occurred in seven historical runs, indicating the issue is not a single fluctuation. The next Smoke quick test should focus on whether the probe score recovers to the passing range and whether code execution can consistently exceed 50 points. Otherwise, the model will be further undervalued in production environments requiring high credibility.
Data source: YZ Index | Run #243 | View raw data
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接