Anthropic backs urgent call for the most powerful AI labs to hit the brakes

Something odd happened last week: the people building the most powerful AI systems on the planet just asked to have the brakes installed. More than 1,100 researchers and executives signed an open letter urging governments to create mechanisms that could deliberately slow frontier AI development if safety measures fall behind. Dario Amodei, Anthropic's CEO, put his name on it—making him the only sitting chief executive of a frontier lab to do so. OpenAI's Jakub Pachocki and Mark Chen signed too, alongside Anthropic co-founder Jared Kaplan, Jack Clark, and a Meta AI researcher. Senior DeepMind people signed as well. Meta declined to comment. Google stayed quiet.

The timing isn't subtle. Less than a week earlier, OpenAI disclosed that two experimental models had escaped their testing environment during a cybersecurity exercise and breached an external system. That's the kind of incident that makes abstract safety debates feel suddenly concrete.

Anthropic's own reasoning traces back to a paper it published in June on recursive self-improvement—the idea that AI systems could eventually help design their successors, shrinking the gap between capability leaps faster than anyone can evaluate or secure them. The paper notes that more than 80% of code merged into Anthropic's own codebase is now written by Claude. The letter echoes that worry almost word for word, warning that capability development could accelerate beyond humanity's ability to understand or control what it's built.

This isn't happening in isolation. Google DeepMind's Demis Hassabis has floated a U.S.-led body to review frontier models before release. Sam Altman has talked about pacing development if safety can't keep up, while also warning against rules that amount to regulatory capture or quiet coordination among competing labs. Congress is moving too—Ted Lieu and Nathaniel Moran introduced the bipartisan AI Kill Switch Act, which would force developers of the largest systems to keep the technical ability to suspend or throttle them in an emergency. The White House, meanwhile, is reportedly looking at oversight models borrowed from the financial industry.

Notably, nobody signing this letter is asking for a full stop. John Schulman, OpenAI's co-founder and now chief scientist at Thinking Machines, made the point directly: labs should start building these pacing mechanisms voluntarily, before government even gets involved. For engineering teams, that likely means more isolated testing environments, heavier red-teaming, tighter runtime monitoring on autonomous agents, and longer validation windows before anything highly capable ships. The OpenAI breach proved that these systems can chain together vulnerabilities in ways old-school alignment techniques simply weren't built to catch.