GPT-6 Astra First Test: YZ Index 18-Question Targeted Evaluation

GPT-6 Astra First Test: 18-Question Targeted Overall 76, Execution 85.7

OpenAI officially announced yesterday that two team members worked with GPT-6 Astra to complete the design of a prototype device for conversing with Codex in less than a week. The project was a side task, and Astra independently completed all aspects of the enclosure, circuit board, interface, and software. The news quickly drew industry attention, and YZ Index immediately launched an 18-question targeted evaluation, twice the number of questions in the routine Smoke test.

This first test used an 18-question rapid targeted format. GPT-6 Astra scored 76 overall, 85.7 in execution, 64.1 in solidity, and pass on integrity. Execution 85.7 shows that the model responded precisely in prototype hardware collaboration and rapid iteration tasks, with outstanding performance in circuit board and interface design; solidity 64.1 reflects room for improvement in software stability and long-term compatibility; integrity pass indicates no obvious fabrication or deviation from facts. The evaluator explicitly stated that this 18-question targeted evaluation is only a temporary benchmark for rapid response to the official update, and the complete baseline will be subject to the next weekly Full evaluation.

When making a rough comparison with the current top of the main leaderboard, the difference in methodology must be emphasized. The most recent complete weekly evaluation on the main leaderboard showed claude-opus-4.7 in first place with 82.5, followed by gpt-6-astra with 82.4, then doubao-pro79.6, gpt-6.1-sol79.3, and grok-4 79.1 in order. The 18-question targeted overall score of 76 differs clearly from the main leaderboard's 82.4, mainly due to differences in question count, dimension coverage, and evaluation cycle, and the two cannot be directly mixed or compared. The main leaderboard uses a full weekly methodology, while this targeted test focuses on the hardware collaboration scenario mentioned in the official announcement, emphasizing execution and rapid prototyping capability.

Next, YZ Index will follow the regular weekly Full evaluation process to conduct complete baseline coverage of GPT-6 Astra, incorporating more dimensions and a broader question set to provide a more reference-worthy complete ranking. If there are new official features or iterative updates, the targeted evaluation will also follow up accordingly.