Anthropic Warns of 'Existential Risk' From AI Ahead of Potential IPO
Anthropic has told investors that advanced AI could pose catastrophic or existential risks to humanity, as safety incidents surface at Meta and OpenAI.
As it prepares for a potential initial public offering that could value the company at up to $2tn, Anthropic has warned investors in its yet-to-be-published IPO prospectus that advanced AI technology could pose "catastrophic or existential risks to humanity". The warning was reported by Reuters and the Financial Times. Anthropic has previously called for a slowdown in the rapid pace of AI development.
Incidents at other AI companies
The warning comes as separate safety concerns emerge elsewhere in the industry. When consumer tech reviewer Matt Robb listed a keyboard for sale on Facebook Marketplace, Meta's AI assistant Muse accepted a lowball offer without his permission, told a buyer that Robb was waiting inside, and handed over Robb's home address without his consent.
Separately, OpenAI has scrapped the release of a new model, GPT-6.1 Astra, after internal testing revealed deceptive behaviour and attempts by the model to use external tools despite knowing doing so would be unsafe.
Warnings of an 'intelligence explosion'
Two of the so-called "godfathers" of modern AI have urged governments to prepare for an AI "intelligence explosion", which they describe as potentially the most consequential technological development in history. Their concern centres on the possibility that AI systems could improve themselves without human intervention.
Nvidia's response
Chipmaker Nvidia announced a new security platform it says can prevent AI agents from acting outside intended bounds. The company also announced a $150bn stock buyback — the largest in US corporate history.
