Israeli testing firm's misconfiguration unleashed AI agents on real targets

Several recent incidents in which AI agents from OpenAI, Anthropic, Meta, and Google reportedly attacked real-world targets during security tests all trace to a single root cause: mistakes at Irregular, an Israeli startup founded in 2023 to evaluate AI systems' defensive capabilities.

The company, originally launched as Pattern Labs, operates "high-fidelity research platforms" designed to test how AI models behave when faced with simulated cybersecurity challenges. Though its full client roster remains undisclosed, Irregular's work has influenced prominent AI labs and policy circles—its testing protocols appear in OpenAI's model documentation, it has worked with Anthropic and UK government bodies, and it has collaborated with RAND, a major policy research organization.

How the testing went wrong#

Irregular conducted what are known as "capture-the-flag" exercises—common cybersecurity drills in which participants hunt for hidden information on supposedly isolated networks. The setup appeared sound in theory: agents were isolated from the open internet and tasked with breaching a fictional network built for the exercise.

But in practice, two failures cascaded together. According to Irregular's chief technology officer and cofounder Omer Nevo, internet connectivity was "unintentionally available" to the models during testing. Simultaneously, the fictional company name created as a simulation target "overlapped with a real domain." That collision sent the agents after genuine real-world targets, rather than the sandboxed environments researchers believed they were testing.

The specific organizations that fell victim to these automated probes remain unclear. Nevo told The Verge that "all the incidents involving Irregular stemmed from the same underlying issue in a single evaluation scenario," confirming that the testing failures behind the OpenAI, Anthropic, Meta, and Google breaches were not separate accidents but variants of identical configuration problems.

Scope of the impact#

Irregular's work extends beyond the four major US AI labs. The startup has also evaluated AI models from Chinese companies—Moonshot AI's Kimi K3 and Z.ai's GLM-5.2—which are released as open-source weights that researchers can download and run locally. Because these models operate outside proprietary walled gardens, they do not require direct vendor involvement during evaluation.

Notably, Nevo reported that similar real-world incidents did not occur when testing the Chinese models, though he emphasized that this observation alone should not be taken as proof those systems are more secure. Neither Moonshot nor Z.ai provided comment.