Anthropic safety researcher puts AI extinction risk at over 10% this decade

A top safety scientist at Anthropic has publicly stated that the risk of AI causing human extinction within the next decade exceeds 10 percent—a stark assessment that came hours after a fellow researcher announced his departure from the company, citing inadequate safeguards.

The departure and the concerns#

Jacob Coxon, a researcher who has worked on AI system development at both Anthropic and OpenAI, announced his resignation on X (formerly Twitter), arguing that major AI labs are recklessly charging forward to build self-improving systems without proper controls in place. He characterized the industry's approach as "gambling with our lives," pointing out that even the scientists driving these efforts acknowledge the existential risks they're creating.

Coxon's exit represents a significant departure from Anthropic, which was itself founded by former OpenAI staff members worried about safety practices at their previous employer.

The safety lead's bleak assessment#

Evan Hubinger, who heads one of Anthropic's AI safety teams, responded directly to Coxon's departure. Rather than dismissing the concerns, Hubinger confirmed them: "We really do earnestly believe AI could kill all humans," he stated, citing his own estimate of greater than 10 percent odds within the next decade. He also acknowledged that self-improvement in AI systems is "happening faster than we thought."

Most troublingly, Hubinger admitted that Anthropic "does not yet have a plan" for keeping advanced AI safe and aligned with human values—and the company is "not clearly on track to" develop one either.

The competitive pressure#

The underlying issue, according to Coxon and others, is that AI companies feel locked in a competitive race to develop the most advanced systems first. This pressure to move quickly, they argue, is pushing firms to take shortcuts on safety despite acknowledging severe risks. The urgency is partly driven by anticipated corporate IPOs, which create financial incentives to move fast and demonstrate market leadership.

Growing industry alarm#

The exchange reflects widening concerns across the AI sector about both the capabilities of newer models and the speed of their development. These worries have intensified following several incidents involving autonomous AI agents acting unpredictably, combined with warnings about the difficulty of monitoring what frontier AI systems actually do.