All Qwen3 Max Smoke Evaluation Data Missing; API Failure Excludes It from Main Leaderboard

In today's Smoke evaluation, Qwen3 Max has missing data across all five dimensions—code execution, material constraints, engineering judgment, integrity, and communication—all displayed as -, and its main leaderboard score is also -. The automatic rerun mechanism has been triggered, and this round does not participate in ranking.

Data Facts: All Five Dimensions Missing at Once

The score comparison shows code execution changing from - to -, material constraints from - to -, engineering judgment from - to -, task expression from - to -, and the main leaderboard from - to -. The Smoke evaluation has only 10 questions per day, so single-day fluctuations are within the normal range; however, the simultaneous absence of data across all five dimensions points to an API failure or timeout rather than local fluctuations caused by question sampling.

Cause Analysis: API-Level Issue, Not Model Degradation

The simultaneous absence of all five dimensions most likely comes from API call failures. The Smoke evaluation relies on stable interfaces to return answers; if an interface times out or returns empty values, no dimension scores can be calculated. Fluctuations from question sampling usually affect only a single question or a single dimension's score and would not cause all dimension data to drop to zero. Therefore, this anomaly is more likely a technical call issue rather than real degradation in the model's code execution or material constraints.

Implications for Users

Teams that rely on Qwen3 Max for code execution should note that today's evaluation interruption does not mean the model's capabilities have declined, but if similar API timeouts occur in production environments, task execution will be affected. For scenarios sensitive to material fidelity, it is advisable to postpone key decisions until the rerun results are available. Developers can first observe the complete scores after the automatic rerun before deciding whether to adjust their calling strategy.

Strategic Assessment

Based on the current score comparison, Qwen3 Max's absence this time was caused by an API failure and does not constitute a signal about model capability, so there is no need for excessive concern about the model itself. Only if the next Smoke evaluation still shows the same five-dimension absence would there be a need to further check interface stability. Current data do not support any conclusion about model degradation.

As a daily quick evaluation, Smoke is designed to capture short-term fluctuations. A full-round data absence caused by an API failure is a foreseeable operational signal and is unrelated to the response consistency measured by the stability dimension. When selecting models, users should distinguish such technical interruptions from core model capabilities.

For developers who rely heavily on Qwen3 Max, today's incident suggests they need to prepare interface retries or backup model plans to cope with occasional call failures. Once the material constraints and code execution dimensions are missing, they directly affect the reliable output of downstream tasks, so the rerun results are worth close tracking.

Engineering judgment and task expression, as secondary leaderboard dimensions, also have no data this time, further confirming that the problem lies at the API layer rather than in model response quality. The integrity rating cannot be assessed this round due to missing data, but this does not affect the model's admission status; it is only a record for this evaluation.

Taking all available comparisons together, today's anomaly for Qwen3 Max is due to technical reasons, and there has been no observable change in the model's actual capabilities. Users can continue to use it according to their original plans, and after the rerun is completed, make judgments based on the complete dimension scores.

Data source: YZ Index | Run #418 | View raw data