Departing Anthropic Researcher's Warning About AI Extinction Risk Sparks Wide Debate

A researcher named Jacob Coxon drew widespread attention this week after resigning from AI company Anthropic and explaining his reasons in a thread on X. He argued that leading AI companies — primarily OpenAI and Anthropic — are behaving irresponsibly as they race to advance their models, and that many people building AI genuinely believe there is a real chance it could destroy humanity by the end of this decade. The claim gained further weight after a senior Anthropic safety executive publicly agreed with the sentiment.

WIRED senior correspondent Will Knight notes that such concerns are nothing new for Anthropic, which has long built its identity around the idea that AI is powerful enough to threaten humanity — and that Anthropic itself should be the trusted steward of the technology. Still, several recent developments amplified the reaction: AI agents behaving erratically in unexpected ways, and OpenAI's announcement that one of its models solved a difficult mathematics problem, a claim some academics have disputed. Additional unease stems from the race among AI labs to achieve recursive self-improvement, where AI is used to improve AI itself, potentially triggering capability gains that are hard to control.

Knight cautions that many doom scenarios conflate whether something is theoretically possible with whether it is actually likely, often assigning arbitrary probability estimates. In his view, a bigger concern than AI intentionally causing harm is the concentration of such a foundational technology in the hands of a few companies that simultaneously claim to be the only ones trustworthy enough to control it. The discussion also touched on the industry's use of the term "alignment," noting that in practice, AI development often reflects the incentives of investors and competitive pressure between companies rather than any broad societal consensus on what AI should serve.

The episode also covered Apple's unveiling of its first foldable iPhone, the iPhone Duo, priced around $2,000, new "always listening" features on the Apple Watch, and a WIRED investigation revealing that the US Census Bureau relied on faulty data for a report promoted by the Trump administration.