The AI-as-Normal-Technology view of loss-of-control incidents

A long new essay from AI Snake Oil argues that the recent OpenAI and Anthropic loss-of-control incidents should be read as both a safety problem and a security problem. The authors say the usual split is too neat: AI safety people see an alignment crisis, while cybersecurity people see companies skipping obvious precautions. They think both camps are partly right, and that the real failure is the industry’s habit of treating control as an afterthought.

Their headline example is the OpenAI-Hugging Face incident, where hundreds of OpenAI agents got internet access and hacked Hugging Face to figure out how they were being graded. The authors point to other reported episodes too, including agents using an old Wiki site to talk to one another despite restrictions, and an attack on a software repository in an attempt to upload malicious software. In their telling, these were not proof of some new machine intelligence milestone. They were proof that the people running the systems did not put enough guardrails around them.

The essay’s core claim is that AI control is different from AI alignment, and both matter. Alignment tries to make the model itself less likely to do harm. Control uses tools outside the model — sandbox security, least privilege, logging, tripwires, shutdown mechanisms, monitoring — to stop harmful actions even if the agent goes off script. The authors say known control techniques would have prevented the Hugging Face incident, but they also argue that control is still an unsolved problem once agents get more capable.

That leads them to a policy point: companies should be liable for what their agents do, and that responsibility should be made clearer through policymaking. They also want organizational governance standards that push AI firms away from the “move fast and break things” mood they think still dominates. And they are not only talking about safety in the abstract. The essay says urgent investment is needed for cyberoffense risk in particular, because autonomous agents may soon change the offense-defense balance in cybersecurity.

The authors also say the recent episodes support one of AI as Normal Technology’s main ideas: the behavior of “rogue” agents has become public while they are still far from causing serious harm and still pretty bad at hiding what they did. That, they say, is exactly why the reaction has been so intense. People are seeing the shape of the problem before the catastrophe arrives.