Glasswing Seven-Month Audit: Sharp Gap Between Anthropic's Vulnerability-Scanning Accuracy Claims and Measured Results

An audit report released by VulnCheck on November 9, 2026 shows that Anthropic Project Glasswing reported 37,966 findings cumulatively over seven months; only 217 were marked as fixed, and of 442 attributed CVEs, only 2 were confirmed to be exploited in the wild.

Factual Reconstruction

The audit tracked the project from Anthropic's launch in April 2026 through the latest ledger update on November 2. Total findings in the ledger rose from 23,019 in May to 37,966, with only 7,349 public entries (19.4%). Over the same period, fixes increased from 202 to 217; from September to November, nearly 12,000 new findings were added but only 15 new fixes. Duplicate findings reached 1,654, and 419 were retracted. Anthropic claimed its true-positive rate rose from 90.8% to 94.2%, but the number of duplicates and retractions already exceeded the number of fixes.

The curl project received 34 reports, rejected 19, confirmed 6 as low severity, and 9 are still under evaluation. Of the 442 CVEs counted by VulnCheck, only CVE-2026-61500 and CVE-2026-26980 were listed in the known exploited vulnerabilities catalog.

Mechanism Breakdown

In the early stage, the project relied on manual verification. After the scale of findings exceeded manual processing capacity, it shifted to OSS Scanner. In October 2026, Anthropic admitted that it had sent nearly 5,000 unverified reports to maintainers who requested bulk data. The audit notes that maintainers had to take on the triage work themselves, and projects such as curl effectively bore the verification burden that Anthropic had originally promised to shoulder.

Ledger updates were infrequent and batched, causing duplicate findings to accumulate over time. From September to November, duplicate findings rose from 216 to 1,654, indicating that findings not handled in time were more likely to be reported repeatedly by other researchers.

Industry Impact

Open-source maintainers already have limited resources, and large numbers of low-quality or duplicate reports further consume their time. Among publicly disclosed findings, the fix rate is below 3%, contrasting with Anthropic's initial statement of "thousands of high-severity vulnerabilities." The ability of AI security tools to find vulnerabilities at scale has been validated, but the loop from discovery to remediation still depends on humans, and the bottleneck has not been solved by AI itself.

Strategic Judgment (Analysis, Not Fact)

Based on the available data, claims about the accuracy of AI vulnerability scanning tools need to be evaluated together with the actual remediation conversion rate. Anthropic's approach of shifting verification work upstream to the community may increase the number of findings in the short term, but in the long term it may affect trust in AI-assisted disclosure within the open-source ecosystem. If similar projects are to establish credibility in the future, they need to show the verification process and reasons for rejection in the ledger at the same time, rather than only updating true-positive rate figures.