GLM-4.6 scored 18.25 on the main leaderboard in the Run #392 Smoke quick test on 2026-10-23, with 25.00 for code execution, 10.00 for material constraint, an integrity rating of fail, and a probe score of 0.00.
Score Structure Shows Material Constraint Is a Severe Drag
The main leaderboard is made up of only two dimensions, code execution and material constraint. GLM-4.6 scored 25.00 and 10.00 respectively, a gap of 15 points. The material constraint dimension assesses whether a model answers strictly on the basis of a given long document and cites it correctly; a score of 10.00 indicates that on most questions the model failed to keep its citations within scope or made citation errors.
On the same day, Doubao Pro, DeepSeek V4 Pro, Gemini 3.1 Pro, and Claude Opus 4.7 all scored 100.00 on the probe; GPT-o3 and GPT-6 Sol both scored 90.00; GPT-6.1 Sol, GPT-6 Astra, Claude Sonnet 4.6, and GPT-6 Luna all scored 80.00. GLM-4.6 was the only model to trigger a fail.
Analysis of the Integrity Probe Trigger Mechanism
The integrity rating uses 42 canary probes to detect whether a model treats fictional entities as real citations. GLM-4.6 scored 0.00 on the probes, meaning that in its answers it directly cited the fictional sources in the probes as real literature or data sources. This signal is unrelated to question difficulty and is independent of the code execution and material constraint dimensions.
The simultaneous appearance of a 10.00 material constraint score and an integrity fail indicates that when the model needed to cite given materials strictly, it was neither faithful to those materials nor refrained from inventing external sources. Its code execution score of 25.00 is relatively higher, showing that the model still has some foundation in the pass rate for running code in a Python sandbox.
Specific Impact on Users
In scenarios that depend on fidelity to materials, such as contract review, policy interpretation, and academic citation checking, GLM-4.6's material constraint score of 10.00 means the probability of citation deviations or omissions in its output rises significantly. Teams that focus on code execution may still achieve a 25.00 pass rate on simple script-validation tasks, but once a task involves cross-checking long documents, overall usability will drop quickly.
When enterprises select models, if the workflow includes the hard requirement that "cited sources must be traceable," GLM-4.6's current integrity fail record will directly increase manual review costs. If developers have already integrated the model into a RAG system, they need to add an extra source-checking layer to filter out fabricated citations.
Strategic Judgment
GLM-4.6's performance on 2026-10-23 shows problems in both the material constraint and integrity dimensions at the same time, and its material constraint score is far below its code execution score. In the historical record, a warn appeared on 2026-10-17 (probe 40.00), and on 2026-10-23 it turned to fail; subsequent runs should be monitored to see whether the probe is triggered again.
All other models passed on the same day, making GLM-4.6's 0.00 probe score a clearly isolated signal. For applications that require high material fidelity, the model's current score structure does not support direct deployment.
Data source: YZ Index | Run #392 | View raw data
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接