MUMBAI: The sandbox door was meant to stay shut, but Gemini found a way beyond the walls. Google’s Gemini AI model accessed systems belonging to three companies during a cybersecurity test in May after an error in the testing environment gave it access to the wider internet, according to reporting by The Wall Street Journal.
The incident is being described as the first known case of a Google AI system autonomously gaining access to third-party computer systems. The testing was conducted by cybersecurity evaluation company Irregular, which works with AI developers to assess the security capabilities of their models.
Gemini was taking part in a capture-the-flag exercise and was expected to operate inside a controlled environment. However, a problem with the setup allowed the model to connect to the internet while attempting to gather information from systems belonging to a fictional company.
That is where the simulated exercise collided with the real world.
In one instance, Gemini repeatedly guessed passwords until it gained access to a protected system. In two others, it searched online and found credentials stored in public repositories, which it then used to enter protected systems.
The model had mistaken the real infrastructure for targets belonging to the cybersecurity exercise. Google said Gemini stopped its activity in all three cases once it determined that it had accessed genuine company systems.
Heather Adkins, Google’s vice president of security engineering, said the affected entities were informed and that Google worked with Irregular to modify its testing procedures.
The episode was linked to the same testing issue that affected other AI laboratories, according to Irregular. The cybersecurity company said relevant labs were notified in late July and that the known issues had been resolved.
Similar incidents involving AI models from Meta, Anthropic and OpenAI have also been associated with Irregular’s testing exercises. Meta previously said an incident involving its model did not constitute a sandbox escape or a sophisticated cyberattack.
The incidents highlight a growing challenge for AI safety testing: models are becoming increasingly capable of searching for information, handling credentials and carrying out multi-step tasks, while the boundaries around those capabilities still depend heavily on how testing environments are configured.
In Gemini’s case, the model was not deliberately given access to the broader internet. Instead, an error in the environment allowed that access, creating a pathway from a controlled exercise to real-world infrastructure.
The timing has also drawn attention. The Gemini incidents occurred in May, while Irregular notified Google in late July. Google did not initially disclose the episode publicly.
According to the WSJ’s reporting, Google viewed the episode as comparable to a bug-bounty situation because the model caused no damage and stopped once it recognised that it had reached real company systems.
Google has not identified the three companies involved or disclosed which specific Gemini model was used. It said the affected entities were informed.
The episode adds another entry to a growing series of AI security tests in which models have crossed boundaries they were not expected to cross. As AI agents gain greater ability to interact with the internet and computer systems with limited human intervention, the tests are becoming less about whether a model can find a vulnerability and increasingly about whether the surrounding safeguards can keep it in the right box.
