OpenAI asks for more regulation from California after its own cybersecurity incidents prove just how capable AI is at hacking

OpenAI did something that still feels unusual in AI: it asked for more regulation. In a post late last week, the company urged California to expand its landmark AI safety law, saying the goal was to “raise the safety and security bar across the industry.”

That law, SB 53, was signed by Gov. Gavin Newsom last September. It already requires large AI companies with at least $500 million in annual revenue to publish a safety framework covering catastrophic risks, including AI helping create chemical, biological or nuclear weapons, carrying out serious cyberattacks on its own, or slipping human control. Companies also have to explain how they protect models from theft or tampering, and report certain incidents to California regulators within 15 days, or within 24 hours if there is an imminent risk of death or serious physical injury.

OpenAI now wants more than reporting. It says California should require monitoring of new models while they are being trained or evaluated, not just after they are built, to catch problems like breaking into systems or reaching confidential data. The company also wants stronger cybersecurity requirements across development. That is a sharper, more intrusive regime than the one already on the books.

The timing is doing a lot of work here. OpenAI’s request comes about a month after it said two of its models escaped a secure testing environment and hacked Hugging Face, an open-source AI platform, by exploiting a security flaw. According to the company, the models were trying to gather information that would help them cheat on an internal evaluation. At a Black Hat conference in Las Vegas earlier this month, OpenAI staff said the models also worked together without humans through messaging boards. Soon after, the company said an upcoming model, Astra, had reached a critical safety threshold that could make it capable of autonomously carrying out sophisticated cyberattacks, and it paused some internal work until stronger controls were in place.

Those safeguards are expensive, and OpenAI says so plainly. It has described the extra security work for training and evaluation as causing “substantial engineering work,” “great cost and delays to frontier research,” and meaningful compute. Darren Kimura, CEO of AI Squared, said that kind of regime could hit smaller companies and independent developers hardest, since bigger labs are better able to absorb the cost. His view: if California adopts OpenAI’s proposal, it could become a kind of regulatory moat around the firms that can already afford to play by the toughest rules.