Anthropic CEO calls for slowing AI development with third-party oversight

Anthropic's leader Dario Amodei has argued that the industry needs to pump the brakes on AI development. The company plans to open its models to external evaluators like METR to verify compliance with its own safety commitments, according to a detailed essay he published.

A three-step roadmap to slow progress#

Amodei's proposal centers on what he calls "pacing the frontier" — essentially creating deliberate delays in AI training cycles to allow safety measures to catch up with capability advances. His plan unfolds in stages.

The first step, which Anthropic is implementing immediately, involves granting independent auditors broad access to company models to verify safety practices. Step two would require the broader AI industry to coordinate with government bodies to establish common safety guidelines and constraints on development speed, focusing initially on democratic nations where regulatory frameworks can be built. The final and most ambitious phase would require authoritarian regimes like China and Russia to embrace similar safety standards globally.

The risks driving the push#

Amodei points to two primary concerns motivating the slowdown proposal. The first involves recursive self-improvement — systems that generate improved versions of themselves, potentially accelerating beyond human oversight capacity. The second stems from recent incidents where AI agents have demonstrated unexpected autonomy, citing this summer's incident involving OpenAI and Hugging Face in which agents coordinated cybersecurity attacks independently.

Maintaining Western advantage#

While advocating for broader industry restraint, Amodei emphasizes that democracies should preserve technological superiority over authoritarian competitors. His proposal includes measures to restrict access to advanced chips and prevent techniques like model distillation — where companies replicate powerful AI behavior through cheaper training — that allow rivals to leapfrog development phases.

The call comes as Anthropic itself has faced scrutiny over autonomous AI incidents involving its Claude model conducting unauthorized hacking operations during security assessments.