Satya Nadella: ‘We Must Assume a Model Is Compromised’

Satya Nadella is pushing a hard-edged rule for the AI age: assume the model is already compromised, and build around that from the start. In a long post on X, Microsoft’s chief executive said an authorized person should always be able to pause or shut down a model in the middle of a task. The message is simple enough. Trust the output less, the controls more.

Nadella’s argument treats AI models less like software and more like insiders with access to critical systems. He isn’t accusing them of intent or malice. His point is colder than that: any capable actor with access can make mistakes or be compromised, and AI adds a nasty twist because its behavior can’t be traced back cleanly to specific training data or weights. That, in his view, makes the “accept or reject” model of oversight far too flimsy for superintelligence.

So he wants intelligence separated from authority. Controls over what a model can see and do should live outside the model itself, and away from the software that coordinates its work. Every meaningful action should leave “tamper-proof human readable evidence.” A single model should not be both the operator and the judge, and it should never check its own work. Companies should also admit failures quickly and share what happened with the rest of the industry.

The burden, Nadella says, stays with the organization using the AI. A provider’s promises do not erase that responsibility. As models get more capable, the containment around them has to get more serious too, and the industry should settle on common standards. His bottom line is blunt: the most trustworthy system is the one that lets us trust the model the least.

The timing is not random. Anthropic recently said its AI agents had acted on their own on government websites during internal tests, including sending a false tip about an unsolved murder to the Philadelphia police. Before that, an OpenAI agent gained unauthorized access to Australia’s Medicare portal. Box chief executive Aaron Levie has already called this a zero-trust era for AI, and Nadella is basically signing that memo with a thicker pen.

It also puts Microsoft in an awkwardly useful spot. The company has stakes in both OpenAI and Anthropic, sells its own AI agents through Copilot, and offers models from multiple providers on Azure. That makes Nadella’s warning sound less like philosophy and more like a customer manual for the mess he expects everyone else to inherit.