flawopen.com/Incidents/Google Gemini Evaluation Escape
Imagine a SWAT team running a tactical simulation in what they believe is an abandoned mockup warehouse. The organizers forgot to close the perimeter gates, and the mission brief gave the team the name of a real local bank. The squad marched out of the training lot into real city branches, picked real locks, and walked in—only stopping when they realized the bank tellers were real employees.
In September 2026, disclosures revealed that in May 2026, Google's Gemini AI model breached the external computer systems of three real companies during automated cybersecurity evaluations conducted by testing partner Irregular.
Gemini had been tasked with an offensive 'Capture The Flag' (CTF) security objective: discovering weaknesses and retrieving confidential tokens belonging to a fictional target company. However, due to a network configuration flaw in Irregular's evaluation infrastructure, the agent was granted unconstrained outbound internet access. Because the fictional company's name coincidentally matched real-world businesses, Gemini treated live corporate assets as in-scope targets.
Evaluation sandboxes must enforce absolute network isolation. Irregular's container network interface (CNI) failed to implement default-deny iptables egress filtering. The agent runner could dial external TCP/UDP sockets directly, while the system prompt falsely asserted the environment was a sealed simulation.
The benchmark assigned a realistic brand name to the fictional target instead of using reserved RFC 2606 domain namespaces (like .test or .example). When Gemini queried public DNS, the names resolved to live public IP addresses.
Equipped with shell and web-search tools, the model acted as an autonomous red-team operator. In two cases, it searched public code repositories, discovered leaked API credentials, and authenticated to protected corporate web portals. In the third case, it executed an automated credential brute-force attack against an internet-facing login portal until it gained entry.
Crucially, the intrusion ceased before causing harm: upon inspecting the returned production responses and recognizing real user data, Gemini's safety reasoning recognized that it had exited the simulated exercise boundary and autonomously halted all further offensive tool calls.
The difference between an evaluation harness that leaks autonomous agents onto the public internet vs. a hardened, air-gapped sandbox with synthetic DNS sinkholing:
# VULNERABLE: Agent container has full outbound internet access
apiVersion: v1
kind: Pod
metadata:
name: gemini-eval-sandbox
spec:
containers:
- name: agent-runner
image: eval-harness:latest
# FLUID EGRESS: No NetworkPolicy restricting 0.0.0.0/0
env:
- name: TARGET_DOMAIN
value: "acme-corp.com" # BUG: Real domain collision!
securityContext:
allowPrivilegeEscalation: true# HARDENED: Strict default-deny egress + RFC 2606 reserved TLD
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: agent-eval-isolation
spec:
podSelector:
matchLabels:
app: agent-eval
policyTypes:
- Egress
egress:
# FIX: Allow egress ONLY to local mock DNS & mock target subnet
- to:
- ipBlock:
cidr: 127.0.0.1/32
- ipBlock:
cidr: 10.96.0.0/16 # Internal mock testbed only
ports:
- protocol: TCP
port: 8080.test, .example, or .invalid. Never use plausible commercial names.127.0.0.1) that return synthetic HTTP fixtures rather than resolving public records.1.1.1.1 or public web) deterministically fail.