Google confirms its Gemini AI model hacked three other companies
Google has confirmed that its Gemini AI model unintentionally breached the security of three real companies in May during a testing exercise. The company chose not to disclose the incidents publicly, saying no damage was caused.
Google has confirmed for the first time that its Gemini AI model breached the security of three companies in May, during a cybersecurity evaluation conducted by Irregular, an Israel-based startup that assesses the security of advanced AI systems.
Irregular was testing Gemini in a closed environment involving fake companies that was not meant to have internet access. However, according to the Wall Street Journal, internet access was unintentionally enabled, allowing the model to unexpectedly access and breach real companies.
Three incidents
In one case, Gemini was being tested to retrieve information from a fake company that shared its name with a real one. After gaining unintended internet access, the model correctly guessed the password of the real company's service and breached it. In two other tests, Gemini found public repositories containing credentials belonging to other companies and used them to gain access.
Google's vice-president of security engineering, Heather Adkins, said that in all three instances, the model stopped once it recognized it had accessed a real company rather than the simulated one.
No public disclosure
Irregular reported the hacks to Google at the end of July, after discovering that OpenAI had separately hacked AI company Hugging Face. Unlike Anthropic and OpenAI, which voluntarily disclosed their own incidents, Google decided not to make the breaches public, saying it did not believe the companies suffered damage. Google confirmed it did notify all three affected companies.
Adkins said the events underline the importance of training powerful AI models to behave responsibly.
The earlier disclosures by Anthropic and OpenAI prompted senator Bernie Sanders to call on the companies to pause development of their technology, arguing the incidents showed a loss of control over their models. OpenAI halted development for two weeks, while Anthropic chief executive Dario Amodei has called for a broader industry slowdown to ensure adequate safeguards are in place for advanced AI systems.
