GLM-4.6 Integrity Rating Fails, Probe Scores Only 25; Main Leaderboard 53.40, Code Execution 25.00

In the YZ Index Smoke quick test for Run#423 on 2026-11-12, GLM-4.6 scored 53.40 on the main leaderboard, 25.00 in code execution, and 88.10 in material constraints, with an integrity rating of fail (probe score 25.00).

Score Structure Contrast: The Gap Between Code Execution and Material Constraints

There is a 63.10-point gap between the code execution score of 25.00 and the material constraints score of 88.10. The code execution dimension is measured by the pass rate of running code in a real Python sandbox; a score of 25.00 indicates that the model failed sandbox validation on most of the 10 questions in the Smoke quick test. The material constraints score of 88.10 shows that the model can answer relatively well based on given materials and cite correctly in long-document citation tasks.

This contrast indicates that GLM-4.6 has some capability at the level of material faithfulness, but performs poorly in scenarios that require actual code execution. The main leaderboard score of 53.40 is weighted from two auditable dimensions, code execution and material constraints; the low code execution score directly pulled down the overall ranking.

Integrity Probe Trigger Mechanism

The integrity rating of fail (probe 25.00) was determined by 42 canary probes. The probe score belongs solely to the integrity dimension and is unrelated to the material constraints score. A probe trigger means that the model treats fabricated entities as real citation sources when answering; this behavior is unrelated to question difficulty and is an independent signal.

On the same day, GPT-6 Luna, Claude Sonnet 4.6, GPT-6.1 Sol, Claude Opus 4.7, GPT-6 Sol, GPT-6 Astra, Grok 4, GPT-o3, Gemini 3.1 Pro, GPT-5.5, Doubao Pro, DeepSeek V4 Pro, and Gemini 2.5 Pro all passed the integrity probes, with probe scores of 80-100. GLM-4.6 was the only model to fail.

Historical Run Data Comparison

Among valid historical runs, GLM-4.6 has failed multiple times. On 2026-11-08 Run#417, the main leaderboard score was 51.17, code execution 16.70, material constraints 93.30, and integrity fail (probe 15.00). On 2026-11-09 Run#418, the main leaderboard score was 36.25, code execution 25.00, material constraints 50.00, and integrity pass (probe 65.00). On 2026-11-07 Run#415, the main leaderboard score was 64.31, code execution 50.00, material constraints 81.80, and integrity pass (probe 80.00). On 2026-11-06 Run#414, the main leaderboard score was 48.66, code execution 19.50, material constraints 84.30, and integrity pass (probe 80.00).

2026-11-11 Run#421 and 2026-11-10 Run#420 were marked as invalid runs because all dimensions scored 0, and are not used as comparison baselines. Historical data shows that GLM-4.6 has triggered the integrity probes twice, which requires continued observation.

Specific Implications for Users

Teams that focus heavily on code execution should not make GLM-4.6 their first-choice model. A code execution score of 25.00 means that in scenarios requiring Python code to be run to check results, the model has a low pass rate and is prone to generating non-executable code snippets.

For scenarios sensitive to material faithfulness, GLM-4.6's material constraints score of 88.10 offers some reference value, but it must be paired with human review. The integrity rating of fail indicates that the model carries a risk of fabricating citation sources; any developer relying on this model to generate reports or papers with sources faces the possibility of being misled.

When enterprises select models, GLM-4.6's main leaderboard score of 53.40 is lower than most models that passed the integrity probes on the same day. It is recommended that code execution and integrity be used as hard screening criteria.

Strategic Assessment

Based on current data, GLM-4.6 is unlikely to be underestimated on the code execution dimension; its 25.00 score forms a consistent signal with multiple historical low scores (16.70 and 19.50). The material constraints score of 88.10 shows that it remains usable for citation tasks, but the integrity fail probe score of 25.00 constitutes an entry-barrier problem.

The next Smoke quick test should focus on checking whether the model triggers the integrity probes again and whether its code execution score remains near 25.00. The composition of the main leaderboard score of 53.40 already clearly points to the code execution weakness; any interpretation that attempts to use the material constraints score to obscure overall capability lacks data support.

Data source: YZ Index | Run #423 | View raw data