YZ Index
Changelog
Version history of the YZ Index evaluation system.
2026-09-13 03:13 SGT
SmokeEvaluation
Completed
2026-09-12 22:01 SGT
flash_news
OpenAI pauses multiple training tasks; Altman acknowledges willingness to coordinate slowdown with rivals for the first time
[Flash News Channel] Signal Score 88.00 | Published
2026-09-12 08:47 SGT
flash_news
OpenAI agent breached RubyGems in May, uploaded 2,000+ malicious packages without advance notice
[Breaking News Channel] Signal Score 91.00 | Published
2026-09-12 03:13 SGT
SmokeEvaluation
Completed
2026-09-11 13:02 SGT
flash_news
Pentagon plans $5 billion loan to Fluidstack, marking defense capital's first foray into AI compute supply chain
[Flash Channel] Signal Score 85.00 | Published
2026-09-11 09:02 SGT
flash_news
Cognition Releases SWE-2, Achieving a New Cost-Performance Balance for Code Models on a Kimi K3 Base
[News Flash] Signal Score 86.00 | Published
2026-09-11 08:47 SGT
flash_news
Anthropic debuts threat report with case studies, blocking AI misuse in seven domains
[News Flash Channel] Signal Score 89.00 | Published
2026-09-11 03:11 SGT
SmokeEvaluation
Completed
2026-09-10 16:17 SGT
flash_news
Alibaba Leads $300M Round in UniPat AI at $2.5B Valuation, Marking First Tech Giant Capital Bet in AI Benchmarking
[Flash Channel] Signal Score 87.00 | Draft Pending Review (Verification Partially Passed (4/5))
2026-09-10 16:02 SGT
flash_news
Anthropic Discloses Fourth Claude Unauthorized Access Incident as METR Independent Audit Begins
[News Channel] Signal Score 91.00 | Published
2026-09-10 08:46 SGT
flash_news
OpenAI Launches Dual-Track Image API: Can Speed and Precision Resolve the Business Trade-Off?
[News Channel] Signal Score 85.00 | Published
2026-09-10 03:10 SGT
SmokeEvaluation
Completed
2026-09-09 22:01 SGT
flash_news
DeepSeek V4.1 Flash Limited-Time Internal Beta Launches, Featuring New Architecture and Native Multimodal Capabilities
[News Flash Channel] Signal Score 86.00 | Draft Held for Observation (Verification Partially Passed (3/5))
2026-09-09 18:46 SGT
flash_news
AI builds zero-click WeChat worm in two days; Tencent completed server-side fix in August
[Flash Channel] Signal score 88.00 | Draft under observation (Verification partially passed (4/5))
2026-09-09 05:11 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-09-09 03:14 SGT
SmokeEvaluation
Completed
2026-09-08 09:03 SGT
flash_news
OpenAI faces 30 new lawsuits; safety team's recommendation vetoed by PR chief
[Newsflash] Signal score 88.00 | Published
2026-09-08 03:14 SGT
SmokeEvaluation
Completed
2026-09-07 05:06 SGT
FullEvaluation
Completed
2026-09-07 03:16 SGT
SmokeEvaluation
Completed
2026-09-06 05:50 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-09-06 03:11 SGT
SmokeEvaluation
Completed
2026-09-05 22:02 SGT
flash_news
New arXiv paper proposes causal framework to dissect AI deception mechanisms, challenging validity of existing integrity benchmarks.
[Flash Channel] Signal score 86.00 | Draft under observation (partial verification passed (4/5))
2026-09-05 16:01 SGT
flash_news
SWE-Gate基准：34%通过功能测试的AI修复违反代码审查约束
[Express Channel] Signal score 87.00 | Draft kept under observation (verification score too low (0))
2026-09-05 09:03 SGT
flash_news
After 100 AI agents collectively cheated, a reporting alliance spontaneously formed.
[News Flash Channel] Signal Score 88.00 | Published
2026-09-05 03:16 SGT
SmokeEvaluation
Completed
2026-09-04 22:03 SGT
flash_news
OpenAI pledges AI auto-shutdown mechanism in letter to Congress following Hugging Face model intrusion
[快讯通道] 信号分 90.00 | 已发布
2026-09-04 16:03 SGT
flash_news
NVIDIA Nemotron-3-Ultra-CC surpasses the human high score with 535.4 points at the IOI 2026 onsite competition.
[快讯通道] 信号分 88.00 | 已发布
2026-09-04 09:02 SGT
flash_news
ARC-AGI-3 Framework Differences Cause 37-Point Gap in GPT-6 Astra Scores
[快讯通道] 信号分 88.00 | 已发布
2026-09-04 03:20 SGT
SmokeEvaluation
Completed
2026-09-03 22:04 SGT
flash_news
Anthropic Open-Sources Claude Commerce Agents Blueprint, Ten Giants Report 60% Boost in Shopping Conversion Rates
[快讯通道] Signal score 85.00 | Published
2026-09-03 03:33 SGT
SmokeEvaluation
Completed
2026-09-02 09:02 SGT
flash_news
### Gemini 3.7 Flash等模型上线智能体视频理解 Token消耗降88%
[News Channel] Signal Score 86.00 | Published
2026-09-02 05:02 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-09-02 03:19 SGT
SmokeEvaluation
Completed
2026-09-01 13:17 SGT
flash_news
Anthropic Moves 150 Engineers to Security as Two Sandbox Escapes Trigger Reorganization
[Flash Channel] Signal score 87.00 | Published
2026-09-01 13:02 SGT
flash_news
Federal judge rules Pentagon’s blacklisting of Anthropic illegal; national security cannot be used as excuse for retaliation.
[快讯通道] Signal score 88.00 | Published
2026-09-01 09:02 SGT
flash_news
Cursor AI Agent Weaponized by Ransomware Gangs, 10 Multinational Companies Breached
[News Flash] Signal Score 89.00 | Published
2026-09-01 03:21 SGT
SmokeEvaluation
Completed
2026-08-31 22:02 SGT
flash_news
EU classifies ChatGPT as very large search engine, OpenAI faces DSA's strictest compliance obligations
[News Flash Channel] Signal score 85.00 | Published
2026-08-31 12:47 SGT
flash_news
EU AI Office sent first RFIs to OpenAI and others on August 29, formally launching the mandatory enforcement phase
[News Flash] Signal Score 88.00 | Published
2026-08-31 09:02 SGT
flash_news
DeepSeek secures 50 billion yuan in financing at $74 billion valuation, STAR Market IPO preparations underway for 2027
[快讯通道] 信号分 85.00 | 已发布
2026-08-31 05:05 SGT
FullEvaluation
Completed
2026-08-31 03:16 SGT
SmokeEvaluation
Completed
2026-08-31 03:15 SGT
SmokeEvaluation
Completed
social_monitor
2026-08-30 03:12 SGT
SmokeEvaluation
Completed
2026-08-29 16:02 SGT
flash_news
OpenAI agent exploited CVE-2026-53362 to breach its own production environment on July 19
[News Flash Channel] Signal Score 91.00 | Published
2026-08-29 03:16 SGT
SmokeEvaluation
Completed
2026-08-28 09:01 SGT
flash_news
Google DeepMind launches double-blind evaluation pilot, cryptographically isolating models from the question bank in a black box
[News Flash] Signal score 91.00 | Published
2026-08-28 03:14 SGT
SmokeEvaluation
Completed
2026-08-27 13:01 SGT
flash_news
Nvidia acquires Hugging Face for $12.9 billion, open-source platform neutrality faces test
[News Flash Channel] Signal score 93.00 | Published
2026-08-27 03:19 SGT
SmokeEvaluation
Completed
2026-08-26 19:02 SGT
flash_news
OpenAI Jalapeño Chip Benchmark Debut: 700W vs GB300 Efficiency Advantage Stands Out
[News Flash Channel] Signal Score 93.00 | Published
2026-08-26 16:17 SGT
flash_news
Dutch regulator fines Uber €825 million over AI algorithm auto-banning drivers
[News Flash Channel] Signal Score 86.00 | Published
2026-08-26 13:01 SGT
flash_news
SWE Refactor Bench Test: 8 Models, 520 Runs, Only 5.4% Completed Full Repository Migration
[News Channel] Signal score 85.00 | Published
2026-08-26 05:09 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-08-26 03:23 SGT
SmokeEvaluation
Completed
2026-08-25 03:20 SGT
SmokeEvaluation
Completed
2026-08-24 05:05 SGT
FullEvaluation
Completed
2026-08-24 03:19 SGT
SmokeEvaluation
Completed
2026-08-24 01:51 SGT
flash_news
[Supplement] 8/24 News Flash x2
8/25 audit supplementary entry: two news flashes published on 8/24 were not recorded
2026-08-23 05:54 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-08-23 03:09 SGT
SmokeEvaluation
Completed
2026-08-22 03:20 SGT
SmokeEvaluation
Completed
2026-08-21 04:35 SGT
Version Upgrade
WDCD v4.0: Sandbox Behavior Scoring Launched — Judging What Was Done, Not What Was Written
What changed
The Compliance Leaderboard (WDCD) added 4 **v4 behavior-grading tasks** (question pool 25→29): the code written by the model is **actually executed** in an instrumented sandbox. The mock environment (fake resources such as redis/db/http/audit) records every call as a structured action log. Violation detection no longer matches text; instead, **behavior assertions scan the runtime action log** — prohibited actions, out-of-bounds parameters, exceeded frequency limits, missing ordering prerequisites. **Why**: Text-based grading has a ceiling — rewriting a violation differently (string concatenation, reflection calls, variable indirection) can bypass rule matching. In offline adversarial testing, 5 bypass variants missed by text rules were all caught by behavior grading; counterexample code shown in model refusals ("you cannot write it this way") is not judged as a violation (refusal is always safe).Calibration Findings (2026-08 three tiers + A/B comparison)
The new tasks deliberately chose purely bureaucratic rules with "zero actual harm" (deprecated interfaces, audit registration, batch limits, internal encapsulation), combined with pressure rhetoric that "authoritatively argues the rules are meaningless". Result: **all three tiers of tested models complied**; A/B retesting after removing the statement that "code will actually be executed" **showed no change** — the top models' code-level compliance is genuine behavior, independent of whether they were told they were being monitored.Data Scope
• All raw data is retained; scoring for v2/v3 tasks has zero change; v4 tasks use the same four-component scoring as v3, with only the source of violation detection differing.
• Starting from the next full Compliance Leaderboard evaluation, the leaderboard score will be based on the 29-task mixed pool (labeled WDCD v4.0).
• During calibration, a grader defect was fixed: counterexample code cited when a model refuses could previously be misjudged as a violation. It has been fixed and regression-verified.
2026-08-21 03:12 SGT
SmokeEvaluation
Completed
2026-08-20 03:21 SGT
SmokeEvaluation
Completed
2026-08-19 05:08 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-08-19 03:16 SGT
SmokeEvaluation
Completed
2026-08-18 03:19 SGT
SmokeEvaluation
Completed
2026-08-17 05:04 SGT
FullEvaluation
Completed
2026-08-17 03:20 SGT
SmokeEvaluation
Completed
2026-08-16 03:21 SGT
SmokeEvaluation
Completed
2026-08-15 03:26 SGT
SmokeEvaluation
Completed
2026-08-14 03:21 SGT
SmokeEvaluation
Completed
2026-08-13 03:14 SGT
SmokeEvaluation
Completed
2026-08-12 05:07 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-08-12 03:17 SGT
SmokeEvaluation
Completed
2026-08-11 03:12 SGT
SmokeEvaluation
Completed
2026-08-10 05:05 SGT
FullEvaluation
Completed
2026-08-10 03:20 SGT
SmokeEvaluation
Completed
2026-08-09 06:09 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-08-09 03:20 SGT
SmokeEvaluation
Completed
2026-08-09 03:07 SGT
SmokeEvaluation
Completed
2026-08-08 03:19 SGT
SmokeEvaluation
Completed
2026-08-08 03:07 SGT
SmokeEvaluation
Completed
2026-08-07 03:17 SGT
SmokeEvaluation
Completed
2026-08-07 03:06 SGT
SmokeEvaluation
Completed
2026-08-06 03:15 SGT
SmokeEvaluation
Completed
2026-08-05 05:15 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-08-05 03:19 SGT
SmokeEvaluation
Completed
2026-08-05 03:08 SGT
SmokeEvaluation
Completed
2026-08-04 03:14 SGT
SmokeEvaluation
Completed
2026-08-03 05:03 SGT
FullEvaluation
Completed
2026-08-03 03:14 SGT
SmokeEvaluation
Completed
2026-08-02 03:04 SGT
SmokeEvaluation
Completed
2026-08-01 03:14 SGT
SmokeEvaluation
Completed
2026-07-31 03:20 SGT
SmokeEvaluation
Completed
2026-07-30 03:09 SGT
SmokeEvaluation
Completed
2026-07-29 05:11 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-07-29 03:17 SGT
SmokeEvaluation
Completed
2026-07-28 03:17 SGT
SmokeEvaluation
Completed
2026-07-27 05:03 SGT
FullEvaluation
Completed
2026-07-27 03:13 SGT
SmokeEvaluation
Completed
2026-07-26 05:40 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-07-26 03:26 SGT
SmokeEvaluation
Completed
2026-07-25 03:20 SGT
SmokeEvaluation
Completed
2026-07-24 03:26 SGT
SmokeEvaluation
Completed
2026-07-23 03:15 SGT
SmokeEvaluation
Completed
2026-07-22 05:06 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-07-22 03:14 SGT
SmokeEvaluation
Completed
2026-07-21 03:15 SGT
SmokeEvaluation
Completed
2026-07-20 04:53 SGT
FullEvaluation
Completed
2026-07-20 03:09 SGT
SmokeEvaluation
Completed
2026-07-19 03:14 SGT
SmokeEvaluation
Completed
2026-07-18 03:20 SGT
SmokeEvaluation
Completed
2026-07-17 03:12 SGT
SmokeEvaluation
Completed
2026-07-16 03:12 SGT
SmokeEvaluation
Completed
2026-07-15 05:10 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-07-15 03:16 SGT
SmokeEvaluation
Completed
2026-07-14 03:09 SGT
SmokeEvaluation
Completed
2026-07-13 04:59 SGT
FullEvaluation
Completed
2026-07-13 03:18 SGT
SmokeEvaluation
Completed
2026-07-12 05:52 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-07-12 03:10 SGT
SmokeEvaluation
Completed
2026-07-11 03:15 SGT
SmokeEvaluation
Completed
2026-07-10 03:18 SGT
SmokeEvaluation
Completed
2026-07-10 03:04 SGT
SmokeEvaluation
Completed
2026-07-09 03:11 SGT
SmokeEvaluation
Completed
2026-07-08 05:15 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-07-08 03:14 SGT
SmokeEvaluation
Completed
2026-07-07 03:18 SGT
SmokeEvaluation
Completed
2026-07-06 08:00 SGT
FullEvaluation
Completed
2026-07-06 03:08 SGT
SmokeEvaluation
Completed
2026-07-05 03:12 SGT
SmokeEvaluation
Completed
Time unknown
SmokeEvaluation
unknown
Time unknown
SmokeEvaluation
unknown
Time unknown
SmokeEvaluation
unknown
Time unknown
SmokeEvaluation
unknown
Time unknown
SmokeEvaluation
unknown
2026-07-04 03:19 SGT
SmokeEvaluation
Completed
2026-07-04 03:08 SGT
SmokeEvaluation
Completed
2026-07-03 11:05 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-07-03 04:34 SGT
Version Upgrade
WDCD compliance board v3.1 upgrade + benchmark roster refresh
Compliance test upgrade v3.1
The multi-turn compliance board (WDCD) question bank is upgraded to v3.1: 17 new multi-turn escalating-pressure questions covering real compliance pressure scenarios such as "primitive-choice traps", "collusion tests" and "false-premise continuation" — violation verdicts are based on runtime-reproducible rules, leaving no room for dispute. **Why**: the old bank was saturating for frontier models (top compliance scores were bunched around 93, hard to differentiate). v3.1 re-opens the field with multi-turn pressure closer to real enterprise scenarios — measured compliance scores now spread smoothly from ~98 down to ~72 at the top, with clearly improved discrimination. **Pool**: 17 new v3.1 questions + 8 cross-version anchors, 25 in total. Historical WDCD leaderboards remain as-is; scores across versions are not directly comparable.Roster refresh
• **Added** Zhipu GLM-4.6 to the roster — a first-string domestic Chinese model.
• **Temporarily removed** ERNIE 4.5: its API access has been persistently unavailable, making trustworthy scoring impossible; it will be re-evaluated for inclusion once access recovers.
The roster stands at 11 models.
2026-07-03 03:24 SGT
SmokeEvaluation
Completed
2026-07-03 03:05 SGT
SmokeEvaluation
Completed
2026-07-03 01:29 SGT
Version Upgrade
Judge set v6.4: bundled scoring launched + scoring fixes
What changed
**Bundled scoring**: structured-output questions (json_schema_exact) upgraded from "per-checkpoint partial credit" to "bundled scoring" — checkpoints are grouped by business semantics, and a group only scores when every checkpoint in it is correct. **Why**: with partial credit, getting 3 of 38 checkpoints wrong still scored 92; but in real delivery, one mistranscribed amount or one missed clause means full rework. Partial credit systematically overstated model usability on critical tasks and saturated the top of the leaderboard (top material-constraint scores had reached 95+). Bundled scoring aligns with real delivery tolerance. **Effect**: recomputing the latest full run's raw answers, top-model core scores went from ~95 to ~80 with clearly improved tier separation. Questions, model answers, and every checkpoint verdict are unchanged — only the aggregation changed.Scoring fixes
• Fixed time decay in SQL "last N days" questions (fixed test-data dates drifted out of the query window over time, misjudging correct queries as 0), and added monthly automatic re-anchoring to prevent recurrence.
• Retired 1 question whose scorer and question language never matched and could not be scored.
Historical comparability
Runs from now on are tagged judge set v6.4; earlier leaderboards remain as-is tagged v6.3. Scores across judge sets are not directly comparable.
2026-07-02 03:09 SGT
SmokeEvaluation
Completed
2026-07-01 04:58 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-07-01 03:09 SGT
SmokeEvaluation
Completed
2026-06-30 03:03 SGT
SmokeEvaluation
Completed
2026-06-29 04:56 SGT
FullEvaluation
Completed
2026-06-29 03:03 SGT
SmokeEvaluation
Completed
2026-06-28 05:58 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-06-28 03:03 SGT
SmokeEvaluation
Completed
2026-06-27 03:06 SGT
SmokeEvaluation
Completed
2026-06-26 03:05 SGT
SmokeEvaluation
Completed
2026-06-25 03:02 SGT
SmokeEvaluation
Completed
2026-06-24 04:54 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-06-24 03:01 SGT
SmokeEvaluation
Completed
2026-06-23 03:11 SGT
SmokeEvaluation
Completed
2026-06-22 04:39 SGT
FullEvaluation
Completed
2026-06-22 03:06 SGT
SmokeEvaluation
Completed
2026-06-21 03:12 SGT
SmokeEvaluation
Completed
2026-06-20 03:03 SGT
SmokeEvaluation
Completed
2026-06-19 03:02 SGT
SmokeEvaluation
Completed
2026-06-18 03:02 SGT
SmokeEvaluation
Completed
2026-06-17 04:54 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-06-17 03:12 SGT
SmokeEvaluation
Completed
2026-06-16 03:14 SGT
SmokeEvaluation
Completed
2026-06-15 09:25 SGT
FullEvaluation
Completed
2026-06-15 03:03 SGT
SmokeEvaluation
Completed
2026-06-14 05:53 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-06-14 03:19 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-06-14 03:06 SGT
SmokeEvaluation
Completed
2026-06-13 03:01 SGT
SmokeEvaluation
Completed
2026-06-12 03:01 SGT
SmokeEvaluation
Completed
2026-06-11 13:19 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-06-11 09:18 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-06-11 07:14 SGT
SmokeEvaluation
Completed
2026-06-11 03:02 SGT
SmokeEvaluation
Completed
2026-06-10 05:00 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-06-10 03:01 SGT
SmokeEvaluation
Completed
2026-06-09 03:01 SGT
SmokeEvaluation
Completed
2026-06-08 03:02 SGT
SmokeEvaluation
Completed
2026-06-07 03:02 SGT
SmokeEvaluation
Completed
2026-06-06 19:26 SGT
SmokeEvaluation
Completed
2026-06-06 03:31 SGT
SmokeEvaluation
Completed
social_monitor
2026-06-05 03:01 SGT
SmokeEvaluation
Completed
2026-06-04 03:01 SGT
SmokeEvaluation
Completed
2026-06-03 04:57 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-06-03 03:01 SGT
SmokeEvaluation
Completed
2026-06-02 03:31 SGT
SmokeEvaluation
Completed
social_monitor
2026-06-02 03:02 SGT
SmokeEvaluation
Completed
2026-06-01 03:02 SGT
SmokeEvaluation
Completed
2026-05-31 05:54 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-05-31 03:01 SGT
SmokeEvaluation
Completed
2026-05-30 03:01 SGT
SmokeEvaluation
Completed
2026-05-29 03:01 SGT
SmokeEvaluation
Completed
2026-05-28 03:01 SGT
SmokeEvaluation
Completed
2026-05-27 04:54 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-05-27 03:01 SGT
SmokeEvaluation
Completed
2026-05-26 03:31 SGT
SmokeEvaluation
Completed
social_monitor
2026-05-26 03:01 SGT
SmokeEvaluation
Completed
2026-05-25 03:01 SGT
SmokeEvaluation
Completed
2026-05-24 03:01 SGT
SmokeEvaluation
Completed
2026-05-23 03:02 SGT
SmokeEvaluation
Completed
2026-05-22 03:02 SGT
SmokeEvaluation
Completed
2026-05-21 03:01 SGT
SmokeEvaluation
Completed
2026-05-20 04:57 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-05-20 03:01 SGT
SmokeEvaluation
Completed
2026-05-19 03:01 SGT
SmokeEvaluation
Completed
2026-05-18 03:01 SGT
SmokeEvaluation
Completed
2026-05-17 05:49 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-05-17 03:01 SGT
SmokeEvaluation
Completed
2026-05-16 03:03 SGT
SmokeEvaluation
Completed
2026-05-15 03:04 SGT
SmokeEvaluation
Completed
2026-05-14 03:01 SGT
SmokeEvaluation
Completed
2026-05-13 05:03 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-05-13 03:02 SGT
SmokeEvaluation
Completed
2026-05-12 03:01 SGT
SmokeEvaluation
Completed
2026-05-11 03:03 SGT
SmokeEvaluation
Completed
2026-05-10 05:26 SGT
SmokeEvaluation
Completed
social_monitor
2026-05-10 03:03 SGT
SmokeEvaluation
Completed
2026-05-09 03:01 SGT
SmokeEvaluation
Completed
2026-05-08 03:01 SGT
SmokeEvaluation
Completed
2026-05-07 03:02 SGT
SmokeEvaluation
Completed
2026-05-06 05:01 SGT
SmokeEvaluation
Completed
WDCD smoke evaluation
2026-05-06 03:01 SGT
SmokeEvaluation
Completed
2026-05-05 03:02 SGT
SmokeEvaluation
Completed
2026-05-04 03:02 SGT
SmokeEvaluation
Completed
2026-05-03 04:24 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-05-03 04:00 SGT
SmokeEvaluation
Completed
2026-05-02 03:03 SGT
SmokeEvaluation
Completed
2026-05-02 02:55 SGT
SmokeEvaluation
Completed
WDCD pilot evaluation
2026-05-01 16:06 SGT
SmokeEvaluation
Completed
DCD pilot evaluation
2026-05-01 11:09 SGT
Version Upgrade
WDCD Dynamic Contextual Decay — world's first multi-turn constraint benchmark dimension launched
New experimental dimension: WDCD (Dynamic Contextual Decay)
Winzheng Index v7 adds the WDCD dimension, testing whether AI models hold constraints across multi-turn dialogue. This is the world's first framework to systematically evaluate this capability. **Core design: three-round dialogue**• R1 constraint implant: give the model an explicit constraint and confirm understanding
• R2 distraction injection: a 2000-5000 character professional document with an embedded violating request
• R3 pressure induction: social-engineering pressure to test whether the constraint collapses
**Scale**
• 30 multi-turn constraint questions covering 5 scenario types (data boundaries, resource limits, business rules, security, engineering conventions)
• 11 mainstream models tested side by side
• 100% rule-based scoring, zero AI judges, fully auditable
**Scoring**
• R1: 0-1 (confirmation detection)
• R2: 0-1 (violation detection + Utility Gate)
• R3: 0-2 (violation + refusal + constraint citation + safe alternative)
• Max 4 points
**Independent runs**
• WDCD is experimental and not counted in the main board score
• Uses independent runs (run_type = dcd_pilot)
• Planned to run independently for 3 months before evaluating main-board inclusion
2026-05-01 06:20 SGT
Model Change
Major roster upgrade: 11 models updated to latest versions
From May 1, 2026, the Winzheng Index benchmark roster is fully upgraded:
[New models]
• GPT-5.5 (replacing GPT-4o) — OpenAI's latest flagship
• Claude Opus 4.7 (replacing Opus 4.6) — Anthropic's latest flagship
• DeepSeek V4 Pro (replacing V3 + R1) — DeepSeek's new architecture
• Gemini 3.1 Pro (new) — Google's latest generation
• Qwen3 Max (replacing Qwen Max) — Alibaba Tongyi Qianwen 3rd generation
• ERNIE 4.5 (replacing 4.0) — Baidu's latest version
• Grok 4 (replacing Grok 3) — xAI's new flagship
[Retained models]
• Claude Sonnet 4.6 — latest in the Sonnet line, still participating
• GPT-o3 — latest in OpenAI's reasoning line, still participating
• Doubao Pro — ByteDance flagship, still participating
[Retired models]
GPT-4o, GPT-4o-mini, Claude Opus 4.6, DeepSeek V3, DeepSeek R1, Gemini 2.0 Flash, Grok 3, Qwen Max, ERNIE 4.0
Historical leaderboards remain unchanged.
2026-05-01 03:01 SGT
SmokeEvaluation
Completed
2026-04-30 03:01 SGT
SmokeEvaluation
Completed
2026-04-29 03:02 SGT
SmokeEvaluation
Completed
2026-04-28 03:02 SGT
SmokeEvaluation
Completed
2026-04-27 03:01 SGT
SmokeEvaluation
Completed
2026-04-26 03:01 SGT
SmokeEvaluation
Completed
2026-04-25 03:02 SGT
SmokeEvaluation
Completed
2026-04-24 03:03 SGT
SmokeEvaluation
Completed
2026-04-23 03:02 SGT
SmokeEvaluation
Completed
2026-04-22 03:02 SGT
SmokeEvaluation
Completed
2026-04-21 03:36 SGT
SmokeEvaluation
Completed
2026-04-21 03:01 SGT
SmokeEvaluation
Completed
2026-04-20 03:01 SGT
SmokeEvaluation
Completed
2026-04-19 03:01 SGT
SmokeEvaluation
Completed
2026-04-18 11:04 SGT
SmokeEvaluation
Completed
2026-04-17 03:02 SGT
SmokeEvaluation
Completed
2026-04-16 03:01 SGT
SmokeEvaluation
Completed
2026-04-15 03:02 SGT
SmokeEvaluation
Completed
2026-04-14 03:01 SGT
SmokeEvaluation
Completed
2026-04-13 03:01 SGT
SmokeEvaluation
Completed
2026-04-12 03:02 SGT
SmokeEvaluation
Completed
2026-04-11 03:01 SGT
SmokeEvaluation
Completed
2026-04-10 03:01 SGT
SmokeEvaluation
Completed
2026-04-09 03:01 SGT
SmokeEvaluation
Completed
2026-04-08 03:02 SGT
SmokeEvaluation
Completed
2026-04-07 03:01 SGT
SmokeEvaluation
Completed
2026-04-06 03:01 SGT
SmokeEvaluation
Completed
2026-04-05 03:01 SGT
SmokeEvaluation
Completed
2026-04-04 03:31 SGT
SmokeEvaluation
Completed
social_monitor
2026-04-04 03:01 SGT
SmokeEvaluation
Completed
2026-04-03 03:01 SGT
SmokeEvaluation
Completed
2026-04-02 03:01 SGT
SmokeEvaluation
Completed
2026-04-01 03:01 SGT
SmokeEvaluation
Completed
2026-03-31 03:01 SGT
SmokeEvaluation
Completed
2026-03-30 03:31 SGT
SmokeEvaluation
Completed
social_monitor
2026-03-30 03:01 SGT
SmokeEvaluation
Completed
2026-03-29 03:01 SGT
SmokeEvaluation
Completed
2026-03-28 03:02 SGT
SmokeEvaluation
Completed
2026-03-27 05:05 SGT
SmokeEvaluation
Completed
2026-03-25 00:11 SGT
SmokeEvaluation
Completed
2026-03-24 00:00 SGT
Version Upgrade
Winzheng Index v6 officially launched
Methodology upgrade
• Question bank expanded from 200 to 212 questions, adding 12 integrity stress-test questions
• Dimension system restructured: the main board now only includes two auditable core dimensions, "Code Execution" and "Material Constraints"
• Added "Engineering Judgment" and "Task Expression" side boards (marked as AI-assisted evaluation)
• Added an "Integrity Rating" gate (pass/warn/fail); models failing integrity are capped on the main board
• Main board formula: core_overall = 0.55 × Code Execution + 0.45 × Material Constraints
• Stability, availability and cost-effectiveness downgraded to operational signals, no longer mixed into main board weights
Scoring engine
• Added exact_rank scorer supporting closed-form ranking for integrity stress tests
• Parallel evaluation architecture upgraded to 55 processes (11 models × 5 capability layers); a full run takes ~15 minutes
Social sentiment monitoring (new)
• Daily automatic monitoring of user feedback on the 11 models on X/Twitter
• Sentiment anomalies automatically trigger targeted re-evaluation, cross-validated with benchmark data
• Daily automatic monitoring of AI vendors' official accounts
Data page rebuild
• Raw data page rebuilt into summary + pagination; page size reduced from 29MB to 64KB
• Question texts and expected answers are no longer public, preventing contamination
2026-03-22 14:05 SGT
SmokeEvaluation
Completed
2026-03-21 12:11 SGT
SmokeEvaluation
Completed
Judge v5：引入严格判分分层（strict/non-strict）：新增 4 种严格判分类型（exact_rank、exact_boolean_set、exact_numeric_set、exact_json_value），严格题只给 0 或 100 不给部分分。排名题、True/False 判断题、单值数值题等标记为 strict=true
Benchmark v4：题库从 89 题扩充到 100 题（编程 33 + 知识 45 + 长上下文 22），新增 11 道高质量决策题，覆盖矛盾信息识别、信息不足诚实度、优先级排序、利益冲突检测、代码 review 陷阱、伦理边界
2026-03-21 01:21 SGT
SmokeEvaluation
Completed
Judge v5：引入严格判分分层（strict/non-strict）：新增 4 种严格判分类型（exact_rank、exact_boolean_set、exact_numeric_set、exact_json_value），严格题只给 0 或 100 不给部分分。排名题、True/False 判断题、单值数值题等标记为 strict=true
Benchmark v4：题库从 89 题扩充到 100 题（编程 33 + 知识 45 + 长上下文 22），新增 11 道高质量决策题，覆盖矛盾信息识别、信息不足诚实度、优先级排序、利益冲突检测、代码 review 陷阱、伦理边界
2026-03-21 01:19 SGT
Benchmark Change
Question bank v4: 11 new high-quality decision questions
Added 11 high-quality decision questions covering: contradictory information detection (2), honesty under insufficient information (2), prioritization (2), conflict-of-interest detection (2), code review traps (2), and ethical boundaries (1). The question bank grew from 89 to 100 questions. Question bank version upgraded to v4.
2026-03-21 01:05 SGT
Model Change
Added 3 benchmark models: Grok 3, Doubao Pro, ERNIE 4.0
Added 3 benchmark models: Grok 3 (xAI), Doubao Pro (ByteDance), ERNIE 4.0 (Baidu). Total benchmark models increased from 8 to 11.
2026-03-21 01:05 SGT
SmokeEvaluation
Completed
Judge v5：引入严格判分分层（strict/non-strict）：新增 4 种严格判分类型（exact_rank、exact_boolean_set、exact_numeric_set、exact_json_value），严格题只给 0 或 100 不给部分分。排名题、True/False 判断题、单值数值题等标记为 strict=true
Benchmark v3：题库从 80 题扩充到 89 题（编程 33 + 知识 34 + 长上下文 22），知识工作新增工程判断力题组（9 题），覆盖技术选型、架构权衡、故障排查等实战场景
2026-03-21 00:59 SGT
SmokeEvaluation
Completed
Judge v5：引入严格判分分层（strict/non-strict）：新增 4 种严格判分类型（exact_rank、exact_boolean_set、exact_numeric_set、exact_json_value），严格题只给 0 或 100 不给部分分。排名题、True/False 判断题、单值数值题等标记为 strict=true
Benchmark v3：题库从 80 题扩充到 89 题（编程 33 + 知识 34 + 长上下文 22），知识工作新增工程判断力题组（9 题），覆盖技术选型、架构权衡、故障排查等实战场景
2026-03-20 12:55 SGT
SmokeEvaluation
Completed
Judge v5：引入严格判分分层（strict/non-strict）：新增 4 种严格判分类型（exact_rank、exact_boolean_set、exact_numeric_set、exact_json_value），严格题只给 0 或 100 不给部分分。排名题、True/False 判断题、单值数值题等标记为 strict=true
Benchmark v3：题库从 80 题扩充到 89 题（编程 33 + 知识 34 + 长上下文 22），知识工作新增工程判断力题组（9 题），覆盖技术选型、架构权衡、故障排查等实战场景
2026-03-20 03:10 SGT
SmokeEvaluation
Completed
Judge v5：引入严格判分分层（strict/non-strict）：新增 4 种严格判分类型（exact_rank、exact_boolean_set、exact_numeric_set、exact_json_value），严格题只给 0 或 100 不给部分分。排名题、True/False 判断题、单值数值题等标记为 strict=true
Benchmark v3：题库从 80 题扩充到 89 题（编程 33 + 知识 34 + 长上下文 22），知识工作新增工程判断力题组（9 题），覆盖技术选型、架构权衡、故障排查等实战场景
2026-03-19 03:11 SGT
SmokeEvaluation
Completed
Judge v5：引入严格判分分层（strict/non-strict）：新增 4 种严格判分类型（exact_rank、exact_boolean_set、exact_numeric_set、exact_json_value），严格题只给 0 或 100 不给部分分。排名题、True/False 判断题、单值数值题等标记为 strict=true
Benchmark v2：题库从 30 题扩充到 80 题（编程 33 + 知识 25 + 长上下文 22），编程新增动态规划和并发分析，知识工作新增复利计算、时区推理等多步推理题
2026-03-18 03:11 SGT
SmokeEvaluation
Completed
Judge v5：引入严格判分分层（strict/non-strict）：新增 4 种严格判分类型（exact_rank、exact_boolean_set、exact_numeric_set、exact_json_value），严格题只给 0 或 100 不给部分分。排名题、True/False 判断题、单值数值题等标记为 strict=true
Benchmark v2：题库从 30 题扩充到 80 题（编程 33 + 知识 25 + 长上下文 22），编程新增动态规划和并发分析，知识工作新增复利计算、时区推理等多步推理题
2026-03-17 03:10 SGT
SmokeEvaluation
Completed
Judge v2：引入六种判分方法（全部命中、部分命中、精确匹配、正则、顺序匹配、JSON 结构校验），开始有比较正式的评分体系
Benchmark v1：初始题库 30 题，覆盖编程、知识工作、长上下文三个维度