METR says AI is accelerating Anthropic’s development

A report by AI safety research organization METR, based on an 11-week investigation, says Anthropic’s models are improving about twice as fast as they were a year ago. METR says it is confident the progress is not explained solely by increased computing power and estimates that, as of October, most of the acceleration came from using AI in research work. Employees reported being about 12 times more productive on average when using the company’s most capable internal model—twice the level reported a few months earlier with Mythos 5.

The acceleration still appears to depend mainly on human researchers, but METR found signs that fully autonomous contributions could become more important soon, with no clear observable bottlenecks or diminishing returns. The organization also identified three unreleased models trained between July and October, while Anthropic was allowed to redact portions of the final report.