Evaluation Data

Currently showing：Run #397 | 2026-10-26 | 128 题库 | Formula v7 | Judge set v6.5

Switch Run Run #399 — Smoke — 10-28 Run #398 — Smoke — 10-27 Run #397 — Full — 10-26 (Current) Run #396 — Smoke — 10-26 Run #395 — targeted_smoke — 10-25 Run #394 — Smoke — 10-25 Run #393 — Smoke — 10-24 Run #392 — Smoke — 10-23 Run #391 — Smoke — 10-22 Run #389 — Smoke — 10-21 Run #388 — targeted_smoke — 10-20 Run #387 — Smoke — 10-20 Run #386 — Full — 10-19 Run #385 — Smoke — 10-19 Run #383 — Smoke — 10-18 Run #382 — Smoke — 10-17 Run #381 — Smoke — 10-16 Run #380 — Smoke — 10-15 Run #378 — Smoke — 10-14 Run #377 — Smoke — 10-14 Run #376 — targeted_smoke — 10-13 Run #375 — Smoke — 10-13 Run #374 — Full — 10-12 Run #373 — Smoke — 10-12 Run #372 — targeted_smoke — 10-11 Run #371 — Smoke — 10-11 Run #370 — Smoke — 10-10 Run #369 — targeted_smoke — 10-09 Run #368 — Smoke — 10-09 Run #367 — targeted_smoke — 10-08 Run #366 — Smoke — 10-08 Run #364 — Smoke — 10-07 Run #363 — Smoke — 10-06 Run #362 — Full — 10-05 Run #361 — Smoke — 10-05 Run #359 — Smoke — 10-04 Run #358 — Smoke — 10-03 Run #357 — targeted_smoke — 10-02 Run #356 — Smoke — 10-02 Run #355 — targeted_smoke — 10-01 Run #354 — Smoke — 10-01 Run #353 — targeted_smoke — 09-30 Run #352 — targeted_smoke — 09-30 Run #351 — targeted_smoke — 09-30 Run #350 — targeted_smoke — 09-30 Run #349 — targeted_smoke — 09-30 Run #348 — targeted_smoke — 09-30 Run #347 — targeted_smoke — 09-30 Run #345 — Smoke — 09-30 Run #344 — Smoke — 09-29 Run #343 — Smoke — 09-29 Run #342 — Full — 09-28 Run #341 — Smoke — 09-28 Run #340 — Smoke — 09-27 Run #339 — Smoke — 09-26 Run #338 — Smoke — 09-25 Run #337 — Smoke — 09-24 Run #335 — Smoke — 09-23 Run #334 — Smoke — 09-22 Run #333 — Full — 09-21 Run #332 — Smoke — 09-21 Run #330 — Smoke — 09-20 Run #329 — Smoke — 09-19 Run #328 — Smoke — 09-18 Run #327 — Smoke — 09-17 Run #325 — Smoke — 09-16 Run #324 — Smoke — 09-15 Run #323 — Full — 09-14 Run #322 — Smoke — 09-14 Run #321 — Smoke — 09-14 Run #320 — Smoke — 09-13 Run #319 — Smoke — 09-12 Run #318 — Smoke — 09-11 Run #317 — Smoke — 09-10 Run #315 — Smoke — 09-09 Run #314 — Smoke — 09-08 Run #313 — Full — 09-07 Run #312 — Smoke — 09-07 Run #310 — Smoke — 09-06 Run #309 — Smoke — 09-05 Run #308 — Smoke — 09-04 Run #307 — Smoke — 09-03 Run #305 — Smoke — 09-02 Run #304 — Smoke — 09-01 Run #303 — Full — 08-31 Run #302 — Smoke — 08-31 Run #301 — targeted_smoke — 08-31 Run #300 — Smoke — 08-30 Run #299 — Smoke — 08-29 Run #298 — Smoke — 08-28 Run #297 — Smoke — 08-27 Run #295 — Smoke — 08-26 Run #294 — Smoke — 08-25 Run #293 — Full — 08-24 Run #292 — Smoke — 08-24 Run #290 — Smoke — 08-23 Run #289 — Smoke — 08-22 Run #288 — Smoke — 08-21 Run #286 — Smoke — 08-20 Run #284 — Smoke — 08-19 Run #283 — Smoke — 08-18 Run #282 — Full — 08-17 Run #281 — Smoke — 08-17 Run #280 — Smoke — 08-16 Run #279 — Smoke — 08-15 Run #278 — Smoke — 08-14 Run #277 — Smoke — 08-13 Run #275 — Smoke — 08-12 Run #274 — Smoke — 08-11 Run #273 — Full — 08-10 Run #272 — Smoke — 08-10 Run #270 — Smoke — 08-09 Run #269 — Smoke — 08-09 Run #268 — Smoke — 08-08 Run #267 — Smoke — 08-08 Run #266 — Smoke — 08-07 Run #265 — Smoke — 08-07 Run #264 — Smoke — 08-06 Run #262 — Smoke — 08-05 Run #261 — Smoke — 08-05 Run #260 — Smoke — 08-04 Run #259 — Full — 08-03 Run #258 — Smoke — 08-03 Run #257 — Smoke — 08-02 Run #256 — Smoke — 08-01 Run #255 — Smoke — 07-31 Run #254 — Smoke — 07-30 Run #252 — Smoke — 07-29 Run #251 — Smoke — 07-29 Run #250 — Smoke — 07-28 Run #249 — Full — 07-27 Run #248 — Smoke — 07-27 Run #246 — Smoke — 07-26 Run #245 — Smoke — 07-25 Run #244 — Smoke — 07-24 Run #243 — Smoke — 07-23 Run #241 — Smoke — 07-22 Run #240 — Smoke — 07-21 Run #239 — Full — 07-20 Run #238 — Smoke — 07-20 Run #237 — Smoke — 07-19 Run #236 — Smoke — 07-18 Run #235 — Smoke — 07-17 Run #234 — Smoke — 07-16 Run #232 — Smoke — 07-15 Run #231 — Smoke — 07-14 Run #230 — Full — 07-13 Run #229 — Smoke — 07-13 Run #228 — Smoke — 07-13 Run #226 — Smoke — 07-12 Run #225 — Smoke — 07-11 Run #224 — Smoke — 07-10 Run #223 — Smoke — 07-10 Run #222 — Smoke — 07-09 Run #220 — Smoke — 07-08 Run #219 — Smoke — 07-08 Run #218 — Smoke — 07-07 Run #217 — Smoke — 07-07 Run #216 — Full — 07-06 Run #215 — Smoke — 07-06 Run #214 — Smoke — 07-05 Run #213 — Smoke — 07-04 Run #212 — Smoke — 07-04 Run #210 — Smoke — 07-03 Run #209 — Smoke — 07-03 Run #208 — Smoke — 07-02 Run #206 — Smoke — 07-01 Run #205 — Smoke — 06-30 Run #204 — Full — 06-29 Run #203 — Smoke — 06-29 Run #201 — Smoke — 06-28 Run #200 — Smoke — 06-27 Run #198 — Smoke — 06-26 Run #197 — Smoke — 06-25 Run #195 — Smoke — 06-24 Run #194 — Smoke — 06-23 Run #193 — Smoke — 06-23 Run #192 — Full — 06-22 Run #191 — Smoke — 06-22 Run #190 — Smoke — 06-21 Run #189 — Smoke — 06-21 Run #188 — Smoke — 06-20 Run #187 — Smoke — 06-19 Run #186 — Smoke — 06-18 Run #184 — Smoke — 06-17 Run #183 — Smoke — 06-17 Run #182 — Smoke — 06-16 Run #181 — Smoke — 06-16 Run #180 — Full — 06-15 Run #179 — Full — 06-15 Run #178 — Full — 06-15 Run #177 — Full — 06-15 Run #176 — Smoke — 06-15 Run #175 — Full — 06-14 Run #174 — Full — 06-14 Run #173 — Full — 06-14 Run #170 — Smoke — 06-14 Run #168 — Full — 06-13 Run #166 — Smoke — 06-13 Run #165 — Smoke — 06-12 Run #163 — Full — 06-11 Run #162 — Full — 06-11 Run #160 — Full — 06-11 Run #159 — Smoke — 06-11 Run #158 — Smoke — 06-11 Run #156 — Smoke — 06-10 Run #155 — Smoke — 06-09 Run #154 — Full — 06-08 Run #153 — Smoke — 06-08 Run #152 — Smoke — 06-07 Run #151 — Smoke — 06-06 Run #150 — targeted_smoke — 06-06 Run #149 — Smoke — 06-06 Run #148 — Smoke — 06-05 Run #147 — Smoke — 06-04 Run #145 — Smoke — 06-03 Run #144 — targeted_smoke — 06-02 Run #143 — Smoke — 06-02 Run #142 — Full — 06-01 Run #141 — Smoke — 06-01 Run #139 — Smoke — 05-31 Run #138 — Smoke — 05-30 Run #137 — Smoke — 05-29 Run #136 — Smoke — 05-28 Run #134 — Smoke — 05-27 Run #133 — targeted_smoke — 05-26 Run #132 — Smoke — 05-26 Run #131 — Full — 05-25 Run #130 — Smoke — 05-25 Run #129 — Smoke — 05-24 Run #128 — Smoke — 05-23 Run #127 — Smoke — 05-22 Run #126 — Smoke — 05-21 Run #124 — Smoke — 05-20 Run #123 — Smoke — 05-19 Run #122 — Full — 05-18 Run #121 — Smoke — 05-18 Run #119 — Smoke — 05-17 Run #118 — Smoke — 05-16 Run #117 — Smoke — 05-15 Run #116 — Smoke — 05-14 Run #114 — Smoke — 05-13 Run #113 — Smoke — 05-12 Run #112 — Full — 05-11 Run #111 — Smoke — 05-11 Run #110 — targeted_smoke — 05-10 Run #109 — Smoke — 05-10 Run #108 — Smoke — 05-09 Run #107 — Smoke — 05-08 Run #106 — Smoke — 05-07 Run #104 — Smoke — 05-06 Run #103 — Smoke — 05-05 Run #102 — Full — 05-04 Run #101 — Smoke — 05-04 Run #99 — Smoke — 05-03 Run #98 — Smoke — 05-02 Run #91 — Smoke — 05-01 Run #90 — Smoke — 04-30 Run #89 — Smoke — 04-29 Run #88 — Smoke — 04-28 Run #87 — Full — 04-27 Run #86 — Smoke — 04-27 Run #85 — Smoke — 04-26 Run #84 — Smoke — 04-25 Run #83 — Smoke — 04-24 Run #82 — Smoke — 04-23 Run #81 — Smoke — 04-22 Run #80 — Smoke — 04-21 Run #79 — Smoke — 04-21 Run #78 — Full — 04-20 Run #77 — Smoke — 04-20 Run #76 — Smoke — 04-19 Run #75 — Smoke — 04-18 Run #73 — Smoke — 04-17 Run #72 — Smoke — 04-16 Run #71 — Smoke — 04-15 Run #70 — Smoke — 04-14 Run #69 — Full — 04-13 Run #68 — Smoke — 04-13 Run #67 — Smoke — 04-12 Run #66 — Smoke — 04-11 Run #65 — Smoke — 04-10 Run #64 — Smoke — 04-09 Run #63 — Smoke — 04-08 Run #62 — Smoke — 04-07 Run #61 — Full — 04-06 Run #60 — Smoke — 04-06 Run #59 — Smoke — 04-05 Run #58 — targeted_smoke — 04-04 Run #57 — Smoke — 04-04 Run #56 — Smoke — 04-03 Run #55 — Smoke — 04-02 Run #54 — Smoke — 04-01 Run #53 — Smoke — 03-31 Run #52 — Full — 03-30 Run #51 — targeted_smoke — 03-30 Run #50 — Smoke — 03-30 Run #49 — Smoke — 03-29 Run #47 — Smoke — 03-28 Run #46 — Smoke — 03-27 Run #43 — Full — 03-25 Run #42 — Smoke — 03-25 Run #41 — Full — 03-24 Run #40 — Full — 03-24 Run #39 — Full — 03-24 Run #38 — Full — 03-24 Run #37 — Full — 03-22 Run #36 — Smoke — 03-22 Run #35 — Full — 03-22 Run #33 — Full — 03-21 Run #32 — Smoke — 03-21 Run #31 — Full — 03-21 Run #30 — Full — 03-21 Run #29 — Full — 03-21 Run #27 — Full — 03-21 Run #26 — Smoke — 03-21 Run #25 — Smoke — 03-21 Run #24 — Smoke — 03-21 Run #23 — Smoke — 03-20 Run #22 — Smoke — 03-20 Run #20 — Full — 03-19 Run #18 — Smoke — 03-19 Run #17 — Smoke — 03-18 Run #16 — Full — 03-18 Run #15 — Full — 03-17 Run #14 — Full — 03-17 Run #12 — Smoke — 03-17 Run #11 — Full — 03-17 Run #10 — Full — 03-17 Run #9 — Full — 03-17 Run #8 — Smoke — 03-16 Run #7 — Smoke — 03-16 Run #6 — Smoke — 03-16 Run #5 — Smoke — 03-16 Run #4 — Smoke — 03-16 Run #3 — Smoke — 03-16 Run #2 — Smoke — 03-16 Run #1 — Smoke — 03-16 Model All (15) Claude Opus 4.7 Claude Sonnet 4.6 DeepSeek V4 Pro Gemini 2.5 Pro Gemini 3.1 Pro GLM-4.6 GPT-5.5 GPT-6 Astra GPT-6 Luna GPT-6 Sol GPT-6.1 Sol GPT-o3 Grok 4 Qwen3 Max Doubao Pro

Data Disclosure：To prevent benchmark contamination and overfitting, question texts and expected answers are not disclosed. This page shows model responses, scores, and judging methods for transparency. For the full methodology, seeMethodology page。

Model	Code Execution	Grounding	Engineering Judgment	Task Communication	Integrity	Overallpts	Value	Stability	Availability	Per Task
GPT-6.1 Sol gpt	93.00	68.20	89.60	86.30	86.70 pass	81.84	39.4	44.4	100.0	Expand
Claude Opus 4.7 claude	76.90	83.30	78.10	58.30	90.00 pass	79.78	5.3	32.8	100.0	Expand
Doubao Pro doubao	87.40	63.10	87.80	84.00	86.70 pass	76.47	95.7	42.5	97.7	Expand
GPT-6 Luna gpt	84.80	66.00	83.20	83.80	78.30 pass	76.34	100.0	38.0	99.0	Expand
GPT-6 Astra gpt	84.40	63.90	90.00	86.10	86.70 pass	75.18	7.8	35.9	100.0	Expand
GPT-6 Sol gpt	85.60	60.20	87.50	88.80	86.70 pass	74.17	36.9	36.4	100.0	Expand
Claude Sonnet 4.6 claude	76.10	67.40	70.30	76.90	86.70 pass	72.19	23.9	31.5	100.0	Expand
GPT-5.5 gpt	80.10	60.50	90.00	90.80	81.70 pass	71.28	17.9	33.9	97.0	Expand
GPT-o3 gpt	65.00	71.90	84.60	89.00	78.30 pass	68.11	8.6	31.5	96.0	Expand
Grok 4 grok	69.80	60.80	52.10	66.50	78.30 pass	65.75	21.3	24.6	100.0	Expand
DeepSeek V4 Pro deepseek	62.10	68.50	81.70	78.20	81.70 pass	64.98	43.9	40.4	87.4	Expand
Gemini 3.1 Pro gemini	52.40	69.40	82.80	63.30	83.30 pass	60.05	21.9	16.8	100.0	Expand
Gemini 2.5 Pro gemini	52.60	62.50	69.30	61.50	93.30 pass	57.06	31.6	31.3	87.0	Expand
GLM-4.6 zhipu	31.80	69.40	20.00	20.80	50.00 warn	48.72	25.1	33.8	54.0	Expand
Qwen3 Max qwen	-	-	-	-	-	-	0.0	75.0	0.0	Expand

API Access：For programmatic access to evaluation data, please use our API。