Doubao Pro Material Constraint Plummets 15.3 Points; Main Leaderboard Drops 4.3 Points

Doubao Pro's Material Constraint score in today's Smoke evaluation fell from 87.90 to 72.60, a drop of 15.3 points, directly causing its main leaderboard score to decline from 89.44 to 85.09.

Core Data Comparison

Code execution rose from 90.70 to 95.30, up 4.6 points; engineering judgment held steady at 100.00; task expression rose from 91.70 to 100.00, up 8.3 points; integrity rating remained pass.

Mechanism Analysis of Score Changes

The Smoke evaluation uses only 2 questions per dimension each day, so a single question error can cause large swings. Material Constraint showed the largest drop this time, indicating that the model's answers clearly deviated from the given material. Code execution improved at the same time, showing that the model's performance on programming tasks did not degrade in tandem, pointing to a problem concentrated in material fidelity rather than overall capability.

Material Constraint 72.60 vs yesterday's 87.90; the single-day change exceeds the normal fluctuation range of code execution.

What This Means for Users

Scenarios that rely on Material Constraint need greater caution. Enterprise RAG systems that need to strictly cite documents, contracts, or knowledge bases may see more hallucinations or deviations from source content in the current version. Developer teams focused on code execution can continue to use it, as the 95.30 score is better than yesterday.

• Material-sensitive scenarios: recommend adding a manual review step
• Code-centric scenarios: existing integrations can be maintained

Strategic Assessment

This 15.3-point drop in Material Constraint exceeds the normal sampling fluctuation range of the Smoke evaluation and warrants continued observation in the next cycle. Engineering judgment remains at full score, indicating that the model's foundational capabilities have not collapsed, but the Material Constraint anomaly needs multiple consecutive days of data to confirm whether it is a real regression.

The main leaderboard score of 85.09 remains in a usable range, but teams sensitive to Material Constraint should temporarily lower priority and wait for subsequent evaluation results.

Data source: YZ Index | Run #381 | View raw data