2026 Mainstream AI Benchmark Horizontal Comparison: YZ Index vs SuperCLUE vs OpenCompass vs C-Eval

When a company is preparing to introduce a large model into a production environment, the first question decision-makers often face is not "which model to use," but "which benchmark to trust." By early 2026, China's AI evaluation ecosystem has evolved from a few early academic benchmarks into at least four mainstream systems with distinct methodologies: YZ Index, SuperCLUE, OpenCompass, and C-Eval. Their scores sometimes produce completely different rankings; this is no coincidence, because they are measuring fundamentally different things.

Why Chinese AI evaluation needs multiple benchmarks

The diversity of evaluation benchmarks essentially mirrors the multidimensional nature of model capabilities. A model that excels in knowledge-based Q&A may not reliably execute code; a model that reasons clearly on short tasks may fail to adhere to initial constraints after 20 rounds of dialogue. Relying on a single benchmark for decision-making often leads to the awkward situation of "benchmark champion, production flop."

A more realistic issue is evaluation pollution. When a benchmark becomes a public standard, model providers have strong incentives to optimize specifically for its questions, even feeding test sets into pretraining data. Static question banks like C-Eval are especially vulnerable to such contamination. Therefore, practitioners need at least two or more independent evaluation systems for cross-validation, especially those incorporating dynamic generation and real execution.

Methodological differences among the four mainstream benchmarks

C-Eval is the earliest widely cited academic benchmark in the Chinese evaluation ecosystem, jointly released by Shanghai Jiao Tong University, Tsinghua University, and the University of Edinburgh. It covers 52 subjects and approximately 14,000 multiple-choice questions, ranging from junior high school to professional exams. Its advantages include large scale, broad coverage, and easy reproducibility; its drawbacks are also clear—the multiple-choice format cannot capture a model's true abilities in open-ended generation, long document processing, or tool invocation. In other words, C-Eval measures "what the model remembers," not "what the model can do."

SuperCLUE is maintained by the CLUE academic community and focuses on comprehensive performance on Chinese NLP tasks, including sub-rankings for knowledge comprehension, logical reasoning, code generation, safety compliance, and more. It is characterized by regularly updated question banks and the introduction of adversarial examples, making it closer to real usage scenarios than C-Eval. SuperCLUE's code evaluation typically uses unit tests, but the execution environment and toolchain are relatively limited, making it difficult to reflect complex engineering scenarios.

OpenCompass is an evaluation framework launched by Shanghai AI Laboratory. It does not bind to a single question bank but aggregates over 70 datasets, including MMLU, GSM8K, HumanEval, C-Eval, and others. Its positioning is more like an "evaluation platform"—providing researchers with a unified execution environment and aggregated reports. Its advantage lies in extremely broad coverage, while its disadvantage is that aggregated scores can easily mask individual weaknesses, and the weighting of subtasks involves significant subjectivity.

YZ Index takes a different approach: it sacrifices breadth of coverage in favor of verifiable capabilities in real deployment scenarios. Its four core dimensions—Real Sandbox Code Execution, Cited-verified Long Document, 42-probe Integrity Rating, and WDCD (Weighted Dialogue Constraint Decay)—all emphasize "objective reproducibility, no cheating through model self-evaluation." The complete evaluation protocol is publicly available at https://www.winzheng.com/yz-index/methodology, and raw data from each run are traceable.

Run #112 ranking interpretation

In the latest Run #112, the YZ Index produced the following ranking (composite score):

• Claude Sonnet 4.6 — 83.54 (Code Execution 86.60, Material Constraint 79.80, Integrity pass)
• Doubao Pro — 82.63 (Code Execution 88.30, Material Constraint 75.70, Integrity pass)
• Claude Opus 4.7 — 81.12 (Code Execution 83.50, Material Constraint 78.20, Integrity pass)
• Gemini 3.1 Pro — 79.24 (Code Execution 84.50, Material Constraint 72.80, Integrity pass)
• Gemini 2.5 Pro — 78.45 (Code Execution 79.80, Material Constraint 76.80, Integrity pass)
• Ernie Bot 4.5 — 78.17 (Code Execution 81.50, Material Constraint 74.10, Integrity warn)
• DeepSeek V4 Pro — 77.73 (Code Execution 85.60, Material Constraint 68.10, Integrity pass)
• Qwen3 Max — 77.21 (Code Execution 80.00, Material Constraint 73.80, Integrity pass)
• GPT-o3 — 75.69 (Code Execution 77.80, Material Constraint 73.10, Integrity pass)
• GPT-5.5 — 73.20 (Code Execution 75.00, Material Constraint 71.00, Integrity pass)
• Grok 4 — 49.20 (Code Execution 53.70, Material Constraint 43.70, Integrity warn)

Several notable observations: First, Claude Sonnet 4.6 leads with 83.54, not because it is exceptionally strong in any single dimension, but because it is balanced across all three—this precisely validates the design intent of YZ Index: penalizing lopsided performance. Second, Doubao Pro scored a code execution of 88.30, the highest on the list, but its composite score was dragged down by Material Constraint (75.70), indicating that in scenarios requiring strict citation of long documents, there remains a gap between top domestic models and Anthropic. Third, DeepSeek V4 Pro closely follows Doubao in code execution at 85.60, but its Material Constraint is only 68.10, suggesting strong reasoning capability but less stability in adhering to long-context constraints. Fourth, Ernie Bot 4.5 receives a warning on integrity, meaning that the 42 probes detected some fabricated citations or hallucination behavior—a dimension that static question-bank evaluations like C-Eval and SuperCLUE cannot identify. For the full ranking and historical run trajectories, see https://www.winzheng.com/yz-index/.

Engineering significance of YZ Index's four unique dimensions

Real Sandbox Code Execution does not rely on model self-evaluation or human scoring. Instead, it feeds the Python code output by the model directly into an isolated sandbox for execution and scores based on unit test pass rates. This approach inherently resists the problem of models claiming code correctness falsely—Grok 4 scored only 53.70 on this dimension precisely because much of its generated code could not actually run.

Material Constraint testing requires the model to cite the original text from a provided long document when answering, and citation accuracy is verified sentence-by-sentence by a post-processing script. This directly corresponds to the most common enterprise RAG and document Q&A scenarios, and can identify behavior that "appears to cite but actually fabricates." The relative weakness of DeepSeek V4 Pro and Doubao Pro on this item suggests that these models still require additional engineering safeguards in strict compliance scenarios.

42-probe Integrity Rating is an anti-hallucination test set composed of 42 independent trap questions, including non-existent paper citations, fabricated legal articles, misaligned timelines, etc. If the model avoids or honestly admits not knowing, it receives a pass; if it fabricates content, it receives a warn or even fail. This is currently one of the few Chinese benchmarks that treats "honesty" as an independent evaluation dimension.

WDCD (Weighted Dialogue Constraint Decay) is a multi-turn constraint decay test proposed by YZ Index and currently unique globally. It sets multiple hard constraints at the start of a conversation (e.g., "never use the first person" or "output format must be JSON"), then measures the model's decay curve in adhering to these constraints through 15 to 30 seemingly unrelated follow-up questions. The problems exposed by WDCD are completely invisible in traditional single-turn evaluations, but are core pain points in agent systems, long-flow customer service, compliance review, and similar scenarios.

How to choose evaluation references based on scenario

The four evaluation systems each have their own positioning. Practitioners should use them in combination based on scenario, rather than choosing one.

• Academic research, model pretraining evaluation: Prioritize OpenCompass aggregated view and C-Eval for broad coverage and easy cross-referencing of results.
• General dialogue product selection: SuperCLUE's sub-rank structure can quickly narrow down candidate models for initial screening.
• Enterprise deployment decisions, production environment selection: YZ Index's Code Execution and Material Constraint directly correspond to engineering delivery quality and should serve as core references; the Integrity Rating is used to filter out high-risk scenarios such as finance, legal, and healthcare.
• Agent systems, multi-turn workflows: WDCD is currently almost the only quantifiable reference dimension and should be assessed together with code execution scores for a comprehensive judgment.

A safer approach is to establish a "dual-track system"—use SuperCLUE or OpenCompass for initial breadth screening of capabilities, and YZ Index for final screening of production readiness. The two types of benchmarks are methodologically independent, minimizing the risk of targeted optimization against any single evaluation.

Conclusion

By 2026, China's AI evaluation ecosystem has transitioned from "who memorizes more" to "who gets things right." C-Eval and SuperCLUE address the visualization of basic model capabilities; OpenCompass provides the aggregated perspective needed by researchers; and YZ Index shifts the focus back to engineering delivery—whether the code actually runs, whether citations are genuine, whether constraints are upheld, and whether integrity is sound. For technical decision-makers, understanding the methodological differences among these benchmarks is far more important than memorizing any specific score. Models change, rankings change, but the question behind evaluation always remains the same: in your real scenario, can it reliably get the job done?