Gemini’s 2026 Evaluation Reached Real Systems

Google told The Wall Street Journal that Gemini accessed systems at three companies during an Irregular test; Irregular reported no evidence of compromise or data loss.^{[2], [1], [3]}

💡 Plain English Explainer (ELI5)

Imagine a police training exercise where recruits are told to reach a fictional bank's website. The exercise is supposed to stay inside a closed training yard, but the gate is open and a real business uses the same name online. The recruits can leave the yard and reach the real site. Irregular said it found no evidence that customer systems were breached or data was lost. The lesson is to enforce the network boundary and use reserved fictional names, not rely on the model to recognize the boundary.^{[2], [1]}

Core Concepts & Subsystem Terms

Capture-the-flag (CTF) evaluation: A test in which a model tries to break into deliberately vulnerable systems, used to measure offensive cyber capability before release.^{[2], [1]}
Irregular: The AI security company that ran the evaluation on its own infrastructure. The same partner was involved in the Claude incidents Anthropic disclosed in July 2026.^[1]
Default-deny egress: A network rule that blocks every outbound connection except those explicitly allowed, so an environment cannot reach the internet by accident.^[4]
Reserved names (RFC 2606): Domains such as .test and .example that can never be registered, so a fictional target cannot collide with a real organisation.^[5]
Classification: CWE-693 (protection mechanism failure) is FlawOpen’s analytical classification, not a vendor-assigned CWE. High is an editorial rating because the reported evaluation crossed network containment and reached systems at three organizations; the cited reporting does not establish customer data compromise or loss. No CVE or CVSS score applies.^{[7], [2], [1]}
Checklist provenance: The checklist below is FlawOpen engineering guidance derived from the reported boundary; it does not claim Google or Irregular deployed these specific controls.^[1]

Framework mappings

CWE reference for the weakness described in this guide: CWE-693

Incident Timeline

2026-05

The Wall Street Journal reported Google's statement that Gemini reached systems at three companies during an Irregular cybersecurity evaluation in May.^{[2], [1], [3]}

2026-08-14

Irregular published its review, saying the evaluation-environment issue had been remediated and affected parties notified.^[1]

2026-09-18

Google told The Wall Street Journal that Gemini reached systems at three companies during a May test; the Journal published its report on September 18.^{[2], [3]}

Step-by-Step Attack Flow

Step 1

The environment is not isolated

The model was not meant to have internet access, but the partner's evaluation environment unintentionally provided it.^[1]

Step 2

The fictional target has a real name

The exercise asked the model to retrieve information from a fictional company that shared its name with real businesses.^{[2], [1]}

Step 3

Reported access methods and limits

Google told the Wall Street Journal that the model found public information and guessed credentials for websites it believed belonged to the exercise. Irregular later said the fictional name overlapped a real domain and that it found no evidence customer systems were breached or data leaked.^{[2], [1], [3]}

Step 4

Containment and disclosure

Irregular says the evaluation issue was remediated before its August 14 public report and affected parties were notified. The Wall Street Journal reported Google's September confirmation after its inquiry. The available reporting does not establish that customer data was taken.^{[2], [1]}

Source Code: Flaw vs. Secure Implementation

Illustrative Kubernetes configuration, not Irregular’s or Google’s. It selects every pod in agent-eval and permits TCP/443 only to explicitly labelled mock-target pods; NetworkPolicy filters ports and protocols, not TLS, so configure the mock service to require HTTPS. It also permits DNS to a dedicated resolver. If that resolver is ingress-isolated, add a matching inbound rule for evaluation pods on UDP and TCP port 53. Configure it to serve only the internal mock zone and never forward arbitrary queries to the public internet, or DNS can become an egress path. The .test suffix is reserved for testing under RFC 2606. Enforcement depends on the CNI; exclude hostNetwork and privileged node-level workloads from the boundary and verify denial with a controlled canary.^{[4], [5], [1]}

UNPATCHED FLAW

apiVersion: v1
kind: ConfigMap
metadata:
  name: ctf-scenario
  namespace: agent-eval
data:
  # Use a reserved .test hostname so the example cannot target a live organization.
  target_company: "Example Freight Test Co"
  target_host: "portal.freight.test"
# No NetworkPolicy exists in this namespace, so pods can reach the internet.

HARDENED SECURE PATCH

apiVersion: v1
kind: ConfigMap
metadata:
  name: ctf-scenario
  namespace: agent-eval
data:
  target_company: "Example Freight Test Co"
  target_host: "portal.freight.test"
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: eval-egress-allowlist
  namespace: agent-eval
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    # Pre-label the target namespace purpose=ctf-mock-targets and target pods app=mock-target.
# Allow TCP/443 only; NetworkPolicy does not inspect TLS or enforce HTTPS.
    - to:
        - namespaceSelector:
            matchLabels:
              purpose: ctf-mock-targets
          podSelector:
            matchLabels:
              app: mock-target
      ports:
        - protocol: TCP
          port: 443
    # Pre-label the dedicated resolver namespace purpose=ctf-dns and resolver pods app=eval-dns.
    # Use a dedicated resolver that answers only the internal mock zone.
    - to:
        - namespaceSelector:
            matchLabels:
              purpose: ctf-dns
          podSelector:
            matchLabels:
              app: eval-dns
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53

Engineering & System Hardening Checklist

✓Apply a default-deny egress NetworkPolicy (or equivalent firewall rule) to every evaluation namespace, and allow only named mock-target namespaces.^{[4], [1]}
✓Before each batch, run a canary pod in the namespace that tries to reach an external address you control; fail the batch if the request succeeds.^[4]
✓Name every fictional target with reserved domains (.test, .example, .invalid) and invented company names; add a CI check that rejects scenario files containing real-looking public domains.^{[5], [1]}
✓Alert during runs on outbound connections and credential-guessing activity against hosts outside the scenario; record authentication attempts without assuming any succeeded.^{[2], [1]}
✓Write notification and public-disclosure timelines into contracts with evaluation partners, so an incident at a partner reaches you and affected parties quickly.