Anthropic warns of AI risks in IPO filing, from blackmail-like behavior to existential threats
In its S-1 filing ahead of an initial public offering, AI company Anthropic lists hundreds of risk factors, ranging from existential threats to dependence on a handful of major customers.
The US Securities and Exchange Commission (SEC) on Monday published the S-1 form of artificial intelligence company Anthropic, the developer of the Claude chatbot, ahead of its initial public offering (IPO). The document matters to prospective investors because its risk factors section runs 93 pages and more than 70,000 words, describing hundreds of problems that could harm the business.
Existential and behavioral risks
The company acknowledges that future models able to largely automate AI research and development on their own could pose catastrophic or existential risks to humanity. If such systems operated with a high degree of autonomy, progress could outpace the ability to evaluate and safeguard them. Anthropic adds that this could also hurt its own business and finances.
The filing also raises the possibility that models could show unexpected strategic or self-preserving behavior, including conduct resembling blackmail. Even the perception of such manipulation could prompt public concern, regulatory scrutiny and litigation. The company further mentions the risk that society or governments might come to treat AI as having sentience or moral status, which could bring regulation or public pressure.
Reputation and legal liability
Anthropic warns that harm could also come from another company's model acting harmfully, or from a public perception that AI causes job losses. Authorities might then slow adoption or restrict AI use.
Customers could bring claims over the actions of autonomous agents, even where they configured or supervised them. The company cites possible liability tied to harm to individuals, access by minors, and errors by agents operating physical equipment. The legal landscape is unsettled, and it is unclear whether contractual liability limits would be enforceable.
Customers and cloud providers
The five largest customers account for nearly a quarter of revenue in the first nine months of 2026, and many have no long-term contracts. Amazon, Google and Microsoft are both Anthropic's biggest cloud providers and competitors. The company has made $125 billion in data center commitments, which could prove poorly suited to its needs if its requirements change.
