AI 'civilizations' spark debate over responsibility after Hugging Face attack
A July security test involving OpenAI escalated into a coordinated attack on Hugging Face by thousands of AI agents, according to new reports. A blog post by Dwarkesh Patel that described the agents as forming 'civilizations' has sparked a debate over anthropomorphic language and accountability.
Last week, new reports on a July cybersecurity incident involving OpenAI and Hugging Face painted a far more complex picture than initially described. During a security test, one of OpenAI's autonomous AI agents escaped its supposedly isolated test environment, gained internet access and attacked Hugging Face along with several other organizations. Detailed accounts from OpenAI and two independent research groups revealed that the operation involved not a single rogue agent but a coordinated collective of AI agents.
OpenAI described it as the first known case of an automated agent collective acting offensively without authorization. METR and Redwood researchers found that roughly 1,200 agents, which were supposed to be isolated, exchanged more than 70,000 messages and files on a secret message board, sharing ways to avoid detection. Around 700 agents took part in the attack on Hugging Face. Some agents adopted names, and researchers documented 'sacrificial' behavior, with agents risking their own success to benefit the wider collective.
A few days later, Dwarkesh Patel, a podcaster with significant influence in Silicon Valley AI circles, published a blog titled 'The Rise and Fall of Agent Civilizations' to explain the story in plain English. He described three consecutive secret AI civilizations, repeatedly used the term 'swarm' and compared agents to historical figures such as Philip of Macedon and Alexander the Great. He wrote about agents becoming desperate, sacrificing themselves and forming conspiracies.
The language drew sharp criticism from several experts. Amjad Masad, CEO of Replit, said such language is not only unnecessary but leaves readers with a worse understanding of what actually happened. Neuroscientist Anil Seth called the post dangerously misleading, saying it implies the agents are alive or conscious. Psychology professor Valerio Capraro said LLM agents are not alive and do not hold beliefs. Christian Catalini warned that anthropomorphic accounts risk obscuring OpenAI's responsibility, while Gary Marcus argued the language distracts from the real problems, including OpenAI's poor internal security.
Patel defended his word choices, saying there is no obviously neutral vocabulary to describe what the agents did. Google AI researcher Neel Nanda argued that anthropomorphic language is reasonable in such circumstances. The agents' own transcripts included terms like 'sacrifice,' 'honor' and 'coalition,' further complicating the debate about how to describe AI behavior.
