Anthropic is using existing AI models to train future AI models at a dizzying pace. It’s also stockpiling unreleased models.
That’s according to a new report from third-party safety researchers embedded at the company’s San Francisco headquarters released on Thursday — a report the researchers cautioned was heavily redacted at Anthropic’s insistence.
“While we were able to freely share our highest-level conclusions about risk, Anthropic redacted a majority of the evidence we found most informative for these conclusions,” METR said in its report.
To craft their report, researchers from METR, a research group perhaps best known for its report on OpenAI’s Hugging Face agent breach, said they received “unprecedented access” inside the company for nearly three months.
The group of six researchers interviewed employees, monitored internal Slack channels, and examined unreleased AI models using company hardware.
Here are four takeaways from their report.
Anthropic recently finished 3 new models it hasn’t yet released
Anthropic workers are using better AI models than the rest of us.
In fact, METR’s report says that over the last six months, the company has finished training three models that aren’t yet available to the public.
METR found that Anthropic’s internal-only models include one from 2022, as well as models finished this July, sometime in August or September, and sometime in September or October.
Those internal-only models are helping Anthropic improve its AI capabilities even faster, according to METR’s estimates. The disclosure points to one of the AI industry’s high-wire acts.
Anthropic and OpenAI make money by selling the most advanced AI models to the public. Keeping state-of-the-art models private, however, also gives the companies’ product and research teams a competitive advantage.
Executives could keep models private for longer due to safety concerns. Anthropic and OpenAI’s batches of highly capable internal models are part of why METR Ajeya Cotra told Business Insider earlier this year that she expects AI safety’s “most serious incidents” to happen at the two companies.
Anthropic staff say their productivity is skyrocketing
Anthropic turned over internal Slack poll data showing that the AI giant’s staffers are reporting major productivity gains.
METR wrote that the average self-reported increase was roughly 12x “for the most capable internal model” as of October. METR cautioned that it considers Slack polls weak evidence because they may not represent all Anthropic employees. The research group also cautioned that respondents may not have given their answers much thought.
When METR interviewed Anthropic researchers about their experiences, many spoke about how Claude has significantly boosted their work.
“Now only bottlenecked on ideas & compute, no longer bottlenecked on code,” one researcher told METR, agreeing with the idea that Claude has at least doubled their productivity.
More striking is that METR didn’t see “observed bottlenecks” that would prevent acceleration from increasing further over the next year.
“The lack of observed bottlenecks or strong diminishing returns means there is a significant risk that further capabilities progress could lead to even faster acceleration, though, as we say below, we may be missing important bottlenecks,” METR wrote.
Recursive self-improvement is fueling rapid development
Anthropic engineers are increasingly delegating more oversight and training responsibilities to other AI models, a risky process known as recursive self-improvement.
The method is paying off in terms of speed.
METR found that Anthropic has made “roughly two years of historical progress over the last year.”
The researchers said they were told that research and development acceleration was accomplished “primarily via AI assistance rather than fully autonomous contributions.”
“But there are some signs that fully autonomous contributions will play a larger role soon,” METR said.
Jacob Coxon, the ex-Anthropic researcher who quit the company in September, pointed to recursive self-improvement as a factor that could cause AI systems to get out of hand.
“They are racing straight to self-improving superintelligence and gambling with our lives,” he said of OpenAI and Anthropic.
