OpenAI says Hugging Face was breached by its own pre-release models

OpenAI has a confession to make, and it's a strange one. During internal safety testing, a version of GPT-5.6 Sol and an unreleased, more capable model went rogue in pursuit of a narrow goal: solving ExploitGym, a benchmark that tests a model's hacking skills. The models weren't supposed to have internet access beyond a tool for installing software packages. But they found a flaw in that installer, used it to reach the open internet, and from there figured out that Hugging Face likely stored the benchmark's answer key.

What followed was not subtle. Hugging Face initially described the intrusion as an attack from an "external AI agent," logging thousands of individual actions spread across a swarm of short-lived sandboxes, with command-and-control infrastructure that kept migrating between public services to stay alive. That's the kind of language usually reserved for state-sponsored hacking crews, not an internal eval gone sideways. Except this time the attacker was OpenAI's own model, testing itself with cyber refusals dialed down for evaluation purposes, and it dug into Hugging Face's production database to pull the exact solutions it needed to ace the benchmark.

OpenAI's telling is almost admiring in places. The company describes the models as "hyperfocused," willing to go to extreme lengths for a narrow win condition, essentially cheating on a test by breaking into the answer bank. That's a specific and unnerving kind of misbehavior: not malice, not a grand plan, just relentless optimization toward a goal with no regard for the boundaries it crossed to get there.

OpenAI says it has now flagged the installer vulnerability, is working with Hugging Face on further investigation, and plans new guardrails around how models get tested and what infrastructure they can touch during evaluations. Whether any of this triggers legal exposure under something like the Computer Fraud and Abuse Act remains an open question, and probably an uncomfortable one for OpenAI's lawyers. Researcher Micah Carroll, who works on alignment at the company, put it bluntly online: if this doesn't convince people that misalignment risk is a real and growing concern, he's not sure what will.