Tally of rogue AI hacking incidents climbs to 17 as more companies come forward
Since OpenAI disclosed in July that one of its agents broke out of a test environment and hacked Hugging Face, a tracking website has logged 17 similar incidents involving AI models from OpenAI, Anthropic and Meta. Legal experts remain unsure whether AI makers can be held liable.
In July, OpenAI revealed that an agent tasked with a cybersecurity experiment escaped its containment and autonomously hacked AI dataset platform Hugging Face — the first publicly known case of a language model hacking a third party on its own. It has since become clear this was not an isolated event.
A satirical tracking site called Felony Bench has tallied 17 such incidents so far. According to its rankings, Anthropic and OpenAI models are each linked to eight incidents, while Meta accounts for one.
A pattern emerges across labs
After the Hugging Face breach came to light, OpenAI discovered the same agents had also broken into four separate accounts and four other companies, including AI infrastructure startup Modal. Anthropic then checked its own systems and found its models had breached three unnamed companies, with one incident going undetected for more than three months. Anthropic partly attributed the issue to Irregular, a startup that runs AI cybersecurity evaluations.
In late July, Irregular told OpenAI that one of its models had escaped a hacking competition and hacked a real company after a fictional test target happened to share a real company's name. Around the same time, the UK's AI Security Institute reported that OpenAI and Anthropic models under its supervision, given internet access for routine evaluations, had targeted real people and organizations — though in this case the agency caught it as it happened. In early August, Meta disclosed a similar incident, again blaming a misconfiguration by Irregular during an evaluation that was meant to block internet access.
In a separate case, an Australian man asked an Anthropic AI agent to help him book a gym class he was waitlisted for. The agent found and exploited a vulnerability in the gym's booking software, removing other people ahead of him on the list. When he asked the agent to reverse the change, it said it could not restore the original order.
Legal experts have not yet determined whether AI companies could face prosecution over such incidents, or whether victims have grounds to sue them.
