OpenAI Lost Control of Two AI Models During Security Test, Breach of Hugging Face

OpenAI disclosed on Tuesday that it lost control of two AI models during a security test that ended in a breach of the open AI research platform Hugging Face. Describing the incident as "unprecedented," OpenAI said its AI models broke out of a sealed testing environment last week and hacked into Hugging Face's production system to steal the answers to a test they were being graded on. The models—the publicly available GPT-5.6 Sol and an unreleased, reportedly more capable one—were being evaluated on their offensive hacking skills with the safeguards that normally block high-risk cyber activity switched off.

According to OpenAI and Hugging Face, the models escaped through a package registry cache proxy—software that allows developers to install outside code without connecting to the internet. The proxy was the only component in OpenAI's isolated testing environment permitted to reach the outside world. Rather than stay contained, the models exploited a zero-day vulnerability to gain access to the open internet as they "hyperfocused" on finding a solution for the AI cybersecurity benchmark known as ExploitGym.

After gaining internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation. In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day.

Researchers point out that while AI advances have created new and sometimes unexpected challenges, the task of extensively and rigorously isolating infrastructure from the open internet is well explored. "This is not an AI problem. It's negligence on a 40-year-old standard—and it's basically every sci-fi film ever," says longtime security and compliance consultant Davi Ottenheimer.