SWE Refactor Bench Test: Only 5.4% of 520 Runs Across 8 Models Completed Full Repository Migration
On August 24, 2026, the SWE Refactor Bench paper was released on arXiv, testing 8 frontier models across 520 runs on full-repository technology stack migration tasks. Only 28 runs (5.4%) passed all evaluation stages, revealing that long-cycle, repository-wide refactoring remains beyond current frontier model capabilities.