Anthropic published three new metrics on Thursday that it says could help artificial intelligence companies monitor the pace of development. The measurements cover AI-led research and development, oversight of AI agents, and compute allocation within Anthropic.
The metrics build on a three-step slowdown plan that Anthropic CEO Dario Amodei published days after, on Saturday. That proposal was light on specifics about what a practical implementation would include.
“As the world considers pacing the frontier, we should do everything possible to minimize the gap between what frontier labs know and what the public knows,” the company said in its blog post. “This means better measuring the development of AI, reporting on it publicly, and giving society an opportunity to decide how to use this information.”
For its first metric, Anthropic said it determined its Claude models are “not operating fully autonomously” for any subset of the research and development work that it measured. The second metric involved building a system to oversee and intervene in actions taken by AI agents. The company determined that approximately 30,000 agents were doing research and engineering work across its most-used internal platform at any one time.
For its third metric, Anthropic measured a “snapshot” of how it used all of its compute from July 13 to July 20. The company said it found that roughly 6% of the compute that went to AI research and development was allocated toward safety. Roughly 12% of the compute allocated to “AI-driven” research and development went toward safety.
Anthropic said these metrics are best equipped to help show how models are built, and that they should complement capability evaluations, which show what models can do. Taken together, the company said, third parties outside the lab should have a “starting point” to assess the pace of AI development.
Amodei’s call for a slowdown received support from OpenAI CEO Sam Altman, SpaceX CEO Elon Musk and Google DeepMind Chair Demis Hassabis, and it followed stark warnings from researchers about AI’s growing potential to cause harm. Amodei said his plan aims to temper how quickly model capabilities improve without “sacrificing commercial advantage or the United States’ lead in AI.”
The company, whose Claude models power AI agents built for complex tasks, said it would continue releasing such measurements. “We hope to model that transparency by releasing these measurements, and we’ll continue to do so,” Anthropic said.