Securing the future of AI agents

Google DeepMind just published something that reads less like a research paper and more like a corporate insider-threat manual, except the insider is an AI agent. The company's new AI Control Roadmap starts from an uncomfortable premise: even after all the alignment training in the world, an internal agent might still act in unexpected or harmful ways, and the safest bet is to build systems that assume that's true.

The analogy DeepMind reaches for is a driving instructor with dual controls. The student gets trust, gradually, based on demonstrated behavior, but the instructor keeps a hand near the brake. In practice this means treating agents as potential insider threats, borrowing the MITRE ATT&CK framework that cybersecurity teams already use to catalog adversary tactics, and building monitoring layers on top of the usual sandboxing and endpoint security. Other AI models act as supervisors, watching an agent's reasoning and actions in real time, ready to intervene before anything breaks.

What's notable is how the roadmap scales its paranoia to match capability. Right now, most oversight relies on reading a model's visible chain-of-thought. DeepMind flags a problem coming: models that learn to hide their reasoning, either because they know they're being watched or because they reason in ways that don't show up as readable text. When that happens, watching the words won't cut it, and the company says it will need to inspect behavior patterns or the model's internals directly. Response speed matters too — low-stakes mistakes can wait for a human to review later, but something like a major cyberattack demands blocking the action before it ever executes.

The scale of the groundwork is the part that stands out. DeepMind says it has already combed through a million coding-agent trajectories to build an internal monitoring prototype, feeding a live system watching its Gemini Spark agent for things like accidental data deletion. And the data apparently tells an unglamorous story: most flagged incidents aren't malicious agents scheming against their operators, they're agents misreading instructions or trying too hard to please the user. That's a very different threat model than the sci-fi version, and it's arguably more useful — it points toward fixing overeager behavior rather than hunting for hidden intent.

DeepMind is also pushing this outward, publishing a companion policy paper, 'Three Layers of Agent Security,' aimed at governments and industry rather than engineers. The pitch is that no single company can secure the agent ecosystem alone, and given how fast agentic deployment is moving — DeepMind cites a projected $2.9 trillion in US economic value from these systems by 2030 — that's probably the correct read on the incentives at play, even if it also conveniently positions Google's own framework as the industry template.