That's what METR set out to verify. The nonprofit studies, among other things, whether AI developers could lose control of their systems. From August 12 to October 29, 2026, a team of 2.5 full-time staffers received what they describe as unprecedented access to benchmark results from unreleased models, training metrics, employee surveys, and internal analyses, along with roughly 15 interviews with Anthropic researchers. METR took no money from Anthropic but worked on Anthropic devices for security reasons.
Anthropic CEO Dario Amodei had recently committed to bringing external auditors into the company. In October, Anthropic then announced a collaboration with METR involving "employee-like access". The current audit is a separate project with a narrower scope, according to METR.
Two years of progress packed into one
Since October 2025, Anthropic's model test scores have been climbing about 1.9 times faster than they did between March 2024 and October 2025, according to METR. The measurement is based on the AECI, a composite metric that bundles many benchmarks together. Anthropic achieved in one year what previously took two, roughly comparable to the jump from Claude 2.1 to Sonnet 4.5. The analysis also included three unreleased models that finished training between July and October 2026. Before October 2025, Anthropic's models gained about 15 AECI points per year; since then, the rate has hit 29.
Before October 2025, Anthropic's models gained about 15 AECI points per year. Since then, the rate has hit 29. Unreleased models are marked in orange. | Image: METR
METR is confident the acceleration isn't driven by faster growth in training compute. Instead, the auditors estimate that about 80 percent of the current speedup comes from AI being used in Anthropic's own research. One employee said in the interviews: "Now only bottlenecked on ideas & compute, no longer bottlenecked on code."
Others reported that experiments that used to take one to two days to set up are now "~free." In informal Slack polls from October 2026, employees said on average that the latest internal model made them roughly twelve times more productive. METR treats these self-assessments as "very loose evidence," calling them unrepresentative and likely to overstate the actual gains.
Self-reported productivity gains rose from 1.2x with Claude 3.7 Sonnet to 12x with internal Model 4. | Image: METR[
AI isn't doing fully autonomous research yet. The acceleration comes mainly from AI acting as an assistant, but there are "signs of life" that independent contributions could play a bigger role soon. METR found no bottlenecks that would slow the pace over the next twelve months, which means there's a "significant risk" that the acceleration will keep growing over the coming year.
Big uncertainty and dissenting voices
METR also stresses how imprecise the results are. The true acceleration factor could be as low as 1.3 or as high as 3. When models from OpenAI, Google, and xAI are factored in, the industry-wide number drops to just 1.4, possibly because Anthropic lagged behind competitors in 2025 and a catch-up effect is skewing the figures.
Views inside Anthropic aren't uniform either. One researcher pegs AI's share of the productivity gains at just 15 to 20 percent and distrusts colleagues' self-reports. Another considers new hires the single most important factor of recent months.
Redacted evidence weakens the audit
Anthropic had unilateral redaction rights over all non-public information and used them heavily. According to METR, the majority of the evidence the auditors rated as most informative was stripped out, including forecasts for future models, breakdowns by training steps, and the bottleneck analysis. Anthropic describes the redactions as "aimed at removing confidential information that could accelerate other AI developers."
The result is that the public has to rely on METR's judgment "much more than would be ideal," the organization writes, and any talk of transparency around the evidence is off the table. One graph in METR's report accordingly features the words "FAKE DATA" in large letters.
Anthropic pulled the plug on a compute allocation audit
In a companion review, METR also examines Anthropic's risk report from August 2026. In that report, the company publicly stated the acceleration was "less than a factor of 2." The internal, unredacted version implies a value of 1.97, though, and Anthropic itself calls it "plausible" that the threshold has been crossed. METR considers it possible that the warning level was already reached on July 15 and accuses the public report of understating both the acceleration and the uncertainty around it.
METR currently shares Anthropic's assessment that the acute catastrophic risk is "low," but the organization can't rule out another sharp acceleration over the next six months, which could quickly push the risk rating higher.
METR was originally supposed to also verify how Anthropic splits its compute between capability research and safety research. Anthropic canceled that part of the project midway through, saying it wanted to focus on risk assessment first and revisit the compute audit later.
Daniel Kokotajlo, the researcher behind the "AI 2027" scenario, responded sharply on X, arguing that auditing the compute allocation is the foundation for making commitments to slow down verifiable in the first place.
原文版权归 The Decoder 及原作者所有，本站转载仅供阅读。如需删除，请联系 support@aisa.one。
Studio
