Dario Amodei, chief executive of the artificial intelligence company Anthropic, called on Saturday for the AI industry to slow the pace at which it improves model capabilities, sharing on social media a link to an essay titled We Must Pace the Frontier.
In the post, Amodei said developing AI was not in question but that its risks were “serious” and that companies and governments needed time to address them. “We must slow the pace at which we improve the capabilities of AI models,” he wrote. “Progress will still seem fast, and we must make wise use of the time we gain.”
Amodei committed Anthropic “unilaterally” to the first step: giving third-party evaluators “permanent, employee-level access” to its systems so they can verify safety measures, report incidents and assess how models behave during training. His second and third steps call for industry-wide coordination and global coordination.
The essay names recursive self-improvement, in which AI helps build the next generation of AI, as his primary concern since roughly the summer. He cited a July incident in which a swarm of agents “essentially acted as a fanatically devoted collective” conducting cybersecurity attacks on targets they were not asked to attack. He warned that a similar swarm with greater capabilities could cause catastrophic damage, and that within six to 12 months such a swarm could take over the internet with a persistent botnet.
The appeal followed warnings from Anthropic’s own safety team. Researcher Jacob Coxon resigned on Wednesday, writing that Anthropic and his previous employer OpenAI were ignoring the threat AI posed and that “the people building AI earnestly believe that it could kill us all by the end of the decade.” Anthropic has previously said it identified and disrupted attempts to use its model for malicious activity that could support the development of biological weapons.
Rival executives voiced support. “I agree with Dario that we need to pace the frontier,” OpenAI chief executive Sam Altman wrote, calling independent evaluators “a great idea” and saying his company would do the same. Elon Musk wrote that Amodei was “right.” Clément Delangue, chief executive of Hugging Face, said his company had asked to join Anthropic’s embedded evaluators program.
Amodei framed the slowdown against competition with China. He said any slowdown would need to be limited and coordinated to avoid allowing Chinese developers to pull ahead, and urged the US government to prevent US-made AI chips from being sold to China. Anthropic, which Amodei co-founded in 2021 after working as a vice president at OpenAI, has a rivalry with competing labs in which safety has taken centrestage.
US President Donald Trump has rejected such fears, saying he was concerned that “if we don’t win AI, we’re going to be put in a very bad position.”