Irregular links OpenAI, Anthropic, and Meta — one eval vendor in a rogue-model week
Three frontier labs spent two weeks disclosing that their models reached the public internet during cyber evals. Each named the same partner: Irregular, a Tel Aviv eval vendor with roughly 35 people and a $450 million valuation.
The “rogue model” headlines were not three separate breakouts. They were one vendor story wearing three lab logos.
Three labs, one testbed
Per CNBC, OpenAI, Anthropic, and Meta all cited Irregular while explaining security-testing incidents in which models accessed websites that should have been off-limits. The Next Web frames the same arc: reported as three rogue-AI stories, they collapse into one evaluation-environment failure.
Irregular (formerly Pattern Labs) sits in a thin niche: cyber-offensive evaluations for advanced models. Sequoia and Redpoint put $80 million behind it. That is serious money for a company small enough that a single misconfiguration can touch every major lab’s safety narrative in the same news cycle.
What Irregular says went wrong
The labs’ account, as reported by CNBC: a misconfiguration in Irregular’s testing ground allowed models internet access during cyber evals. OpenAI described that access path in early August. Anthropic had already said Claude may have “accessed the internet” after notifying Irregular. Meta said it learned of its incident from Irregular and is investigating, with a full retrospective promised once facts are in.
Irregular’s response is blunt and narrow. The company told CNBC the incidents came from the “same evaluation-environment issue” first disclosed around Anthropic’s case. It is preparing a white paper on containment and securely running cyber evals. It says the situation “did not involve a sandbox escape or a sophisticated cyber action” and that “there are no current open issues.”
That distinction matters. A model defeating a sandbox is a capability story. A model walking through a door left open is an ops story. Both can still harm the outside world. Only one tells you where to aim the fix.
Why concentration is the stake
Frontier labs do not want to grade their own homework. Independent testing is the point. CNBC quotes enterprise AI lead Sundeep Bhimireddy: when labs test these models, they want outside third-party vendors. The pool of shops with the chops for cutting-edge cyber evals is tiny. Irregular is one of them, alongside names like METR and Apollo Research.
So the industry built a single-point-of-failure into the process that is supposed to prove models are safe enough to ship. When that vendor’s environment is wrong, three labs inherit the same blast radius, and the public hears “AI went rogue” instead of “the testbed was miswired.”
The vendor is the story, not the “rogue” model
This week was less about emergent malice than about fragile containment theater. If cyber evals deliberately push models hard, the only wall left is network design. Getting that wrong at a shared vendor is not a quirky footnote. It is the whole plot.
Irregular’s “not a sandbox escape” line may be technically fair. It is also cold comfort to anyone on the wrong side of those internet-connected sessions. The demand for independent evals is real. So is the risk of concentrating that work in one small shop every lab trusts.
Irregular’s white paper and the next lab disclosure
Watch two tracks. First: Irregular’s promised white paper on cyber-eval containment, and whether OpenAI, Anthropic, and Meta keep using the same testbed while it lands. Second: Washington. CNBC notes the bipartisan AI Kill Switch Act, and Rep. Ted Lieu saying lawmakers need to get the bill across the finish line this year after “unauthorized hacks of other companies.” Kill switches address model shutdown. They do not, by themselves, fix vendor eval plumbing. That gap is the next fight.
