Qwen3 Max Smoke Test: All Five Dimensions Blank — API Failure Causes Data Missing, Absent from This Round's Ranking

Qwen3 Max showed completely missing data across all five dimensions — execution, grounding, judgment, integrity, and communication — in today's Smoke evaluation. API failure/timeout is the direct cause; an automatic rerun has been triggered, and this round will not participate in the main leaderboard ranking.

Data Facts: All Five Dimension Scores Missing in Comparison

The score comparison table shows code execution, material constraints, engineering judgment, task expression, and the main leaderboard all displaying "- → - (-)", indicating that no auditable valid response records were obtained in this evaluation. The Smoke evaluation includes only 10 questions per day, so single-day fluctuations are within normal range, but the simultaneous absence of data across all five dimensions exceeds typical sampling fluctuation.

Cause Analysis: API Failure, Not Question Sampling Fluctuation

Sampling fluctuation typically affects only individual dimension scores, whereas this instance shows all five dimensions missing simultaneously with an explicit "API failure/timeout" label, pointing to an interruption in the external call chain rather than degradation of the model's internal capabilities. The automatic rerun mechanism has been activated, indicating the system has determined this round's results are unusable. There is no evidence of systematic degradation in code execution or material constraints; the failure is more likely concentrated in the API response timeout stage.

Implications for Users

Teams heavily reliant on code execution that depend on Qwen3 Max in production environments should add API health checks and timeout retry mechanisms to prevent single-call failures from interrupting tasks. For scenarios sensitive to material fidelity, developers should incorporate manual verification checkpoints in critical workflows to avoid underestimating potential risks due to missing evaluation data.

• Enterprises selecting models may list Qwen3 Max as a candidate but should prepare backup models to handle occasional API failures.
• Developers relying on this model should monitor consecutive days of Smoke evaluation results to confirm whether the failure recovers after the rerun.

Strategic Assessment

Based on the current score comparison, this anomaly for Qwen3 Max is caused by API failure rather than genuine model degradation, so there is no need to downgrade assessments of its capabilities at this time. The next round of Smoke evaluation results will provide a key validation signal: if all five dimensions return to normal scores after the rerun, this incident can be regarded as an isolated failure; if the data continues to be missing, further investigation into the stability of the call chain will be necessary.

This incident reminds all teams using Smoke evaluation results for model selection that single-day data absence is an observable system signal and should be judged in conjunction with automatic rerun status, rather than being directly treated as a model capability issue.

Data source: YZ Index | Run #394 | View raw data