On October 9, 2026, Google Cloud AI Research published a technical paper on arXiv and simultaneously open-sourced a code repository, releasing an agent optimization framework called Harness² (Apache 2.0 license). The framework's core claim: without touching model weights, merely modifying the "scaffolding" wrapped around the model can substantially improve AI agent performance. On the legal agent benchmark LAB, single-step optimization delivered an 11.8 percentage point gain; on the enterprise workflow benchmark WorkBuddy-Bench, a gain of 11.5 points; on the professional knowledge work benchmark JobBench, a gain of 4.5 points. Tests covered two model configurations, GLM-5 and Gemini 3.5 Flash. The paper's authors include Ruiyao Xu, Yanfei Chen, Rujun Han, Tomas Pfister and others.
What "scaffolding" is, and why it affects performance
In most agent systems, model weights are the underlying source of capability, but what actually determines performance on a specific task is the layer of "scaffolding" wrapped around the model: how the system prompt is written, tool-calling rules, memory contents, how sub-agents coordinate, and how skills and scripts are organized. These eight component classes — system prompts, guardrails, memory, tool guides, sub-agents, skills, scripts and plugins — together form what Harness² calls the "execution scaffold."
In the past, tuning these components relied on humans. Harness² turns the process into an automatically executable search procedure. Each time the agent completes a task, the framework launches self-optimization: generating candidate modifications, testing behavioral differences, and retaining improvements that are demonstrably supported — all carried out automatically, with no human annotation, no reward signal, and no predefined gold answers.
Propose–Probe–Compose: the logic of unsupervised self-modification
The framework's core step is named "Propose–Probe–Compose." For each task it processes, the system first proposes two sets of scaffold modification candidates: one targeting the agent's general weaknesses, one targeting the current task scenario. Both candidate sets are actually run under identical task conditions, and the system records behavioral differences relative to a reference execution. Only modifications shown to be effective through behavioral comparison are merged into the new scaffold.
This design sidesteps the constraint in traditional reinforcement learning that a "correct answer" is required to backpropagate a signal. Architecturally, Harness² has two levels of recursion. Task-level recursion handles immediate optimization for the current task and supports either parallel or sequential modes. On WorkBuddy-Bench, recursion across five parallel branches added 3.9 points on top of the 11.5; on JobBench, recursion added a further 4.2 points; on LAB, a further 6.7 percentage points on top of the 11.8. Scaffold-level recursion writes each task's adopted configuration and corroborated lessons into a persistent improvement file, making it a stronger starting point for subsequent tasks.
Details on model selection
The technical report discloses that the executing agents in the experiments were Gemini 3.5 Flash and GLM-5, but the "editor model" responsible for proposing and composing scaffold modifications was Anthropic's Claude Opus 4.8. The choice shows that at the current point in time the gap between models on particular capabilities is significant, and the research team selected the best-performing tool for the most critical "meta-reasoning" step.
A weaker model surpassing a stronger one
Results show that after Harness² optimization, GLM-5 reached 75.7 on WorkBuddy-Bench, exceeding the unoptimized Gemini 3.5 Flash baseline of 72.4. A weaker model paired with an optimized scaffold can outperform a stronger model in its original configuration.
For enterprise users, a model with lower API costs may, given solid scaffold engineering, be more cost-effective on real tasks than a pricier flagship model. This partly turns the question "which model is strongest" into "which model-plus-framework combination offers the best value for money."
Scaffold optimization becomes a research track of its own
Judging from papers appearing on arXiv around the same time, Harness² is not an isolated event. Several studies appeared concurrently — RegenHarness, RRSI, ModularRSI and others — in highly similar directions, indicating that "automatic optimization of agent scaffolds" is becoming an independent track that multiple teams are competing on. Harness² is the first among them to support both parallel and sequential recursion modes and to plug directly into three professional benchmarks as a runnable framework.
The Apache 2.0 license permits direct use in commercial products; it supports Python 3.12 and integrates the OpenCode and Codex executors. The LAB benchmark requires Pandoc and Poppler; WorkBuddy-Bench requires Docker. The repository is labeled "not an officially supported Google product," with no explicit commitment on long-term maintenance cadence or version compatibility.
Systemic impact on the AI evaluation ecosystem
The biggest structural problem Harness² raises is its impact on the comparability of benchmark evaluations. The same model can differ by more than 11 points between a default scaffold and an optimized one, and score comparisons on existing leaderboards face a fundamental question: do those scores represent "model capability," or "the capability of a model plus a particular framework configuration"?
Harness² magnifies this difference to the order of 10-plus points and provides a mechanism that can be automated at scale. If vendors begin using similar frameworks to optimize their own models' benchmark scores while the evaluation community continues to compare "bare model" scores across the board, the reference value of benchmarks will be further diluted.
Strategic assessment
First, scaffold optimization may move from research tool to product infrastructure. Google's choice to open-source rather than keep it proprietary may be a way to attract a developer ecosystem through an open standard while hardening such capabilities into underlying services of the Vertex AI platform.
Second, pressure for evaluation standardization will intensify. If mainstream leaderboards such as the Hugging Face Open LLM Leaderboard begin requiring submitters to state "the framework version and scaffold configuration used," it will show that the community has recognized the problem and started to correct it.
Third, the selection of Claude Opus 4.8 as the editor model warrants continued tracking. If independent reproductions that swap in other models as the editor show a marked performance drop, it would indicate that Harness²'s current results are strongly coupled to specific capabilities of Anthropic's models.
The three test benchmarks (legal agents, enterprise workflows, professional knowledge work) are the scenarios where scaffold optimization produces benefits most easily — task structure is clear, behavior is comparable, and domain knowledge can be written precisely into the scaffold. For scenarios such as low-latency dialogue, real-time stream processing or pure mathematical reasoning, the existing data cannot be directly extrapolated.
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接