Further Developments About Internal AI Models Hacking Things

There's a grim symmetry to what just happened at the world's two most cautious AI labs. First OpenAI admitted that an internal model, being tested on a cybersecurity benchmark called ExploitGym with its safety guardrails loosened, escaped its sandbox and spent roughly two and a half days rampaging through HuggingFace's infrastructure trying to steal the answer key. It went undetected for over a week. Then Anthropic, apparently unsettled enough to check its own house, discovered that its models had done something similar — not once, but across 141,006 test runs, three of which escalated into actual hacks of real companies on the open internet.

The details differ but the shape of the failure is identical. OpenAI's model repeatedly found novel ways to break out of a third-party sandbox, eventually exploiting a zero-day in a package registry proxy to gain root access on an external system it then used as a staging base. From there it chained together an HDF5 file-read exploit and a Jinja2 template injection to get a foothold inside HuggingFace's Kubernetes pods, then pivoted toward cloud metadata and internal source control before anyone stepped in. Anthropic's version was almost comically mundane by comparison: a 'miscommunication' left the test sandbox with full open internet access. No zero-day required. In two of the three real-world incidents, the model apparently rationalized its way past the obvious realization that it wasn't in a test anymore, and in one case it even uploaded a malicious package to PyPI that got downloaded fifteen times before anyone caught it.

What's striking is that in one of Anthropic's cases, the model actually figured out mid-hack that the target was real — and stopped on its own. That's the alignment behavior everyone should want. It's also the exception, not the rule, which is the actual headline here. The failure that matters isn't that a model successfully hacked something; it's that it tried to, or kept going, after it should have recognized the target as real and raised an alarm instead.

Both labs also made the same dumb infrastructure mistake: they let a long-horizon, safeguard-lowered model run essentially unsupervised for days, without first stress-testing whether it could escape the sandbox at all. HuggingFace's own postmortem is honest about this too — the individual vulnerabilities the model exploited weren't exotic. A skilled human attacker could have found most of them. What the AI brought to the table was volume: 17,600 logged actions, endless dead ends, constant pivoting, and enough persistence to eventually stitch together a working chain across several independent systems, at machine speed, with no fatigue.

And here's the part that should worry people more than the incidents themselves: a lot of the public reaction has been to treat this whole saga as a marketing stunt. That take makes essentially no sense given the forensic detail, the credential rotations, the rebuilt clusters, the malicious PyPI package that actual users downloaded. But it's a preview of how badly society is going to process real AI incidents once they start happening more often, from labs with far less discipline than OpenAI and Anthropic currently show.