AI researchers call for new tools that can slow automated model development

More than a thousand people who build frontier AI models just put their names on a letter that argues, essentially, for putting the brakes on their own industry's fastest-moving frontier: AI systems designing other AI systems. The tally is 1,134 signatories, and they don't come from the fringes. Anthropic, OpenAI, Google and Meta employees are on the list, alongside people from smaller venture-backed labs like Thinking Machines and London's Inherent Laboratories. Dario Amodei signed, so did several Anthropic co-founders, and they were joined by chief scientists from OpenAI, Meta AI and Thinking Machines.

The worry at the center of the letter is specific: automated AI development, meaning AI systems that help build and improve the next generation of AI systems, could accelerate capability gains faster than anyone's capacity to understand or steer what comes out the other end. The signatories aren't claiming this is certain. They're careful to say it's hard to predict how much automation will speed things up. But they call the risk real enough that the field needs a way to deliberately slow the pace, and they say the tools to do that pacing simply don't exist yet. So the ask is that the U.S. government back an international push to build both the technical and governance mechanisms required.

This isn't a hypothetical concern plucked from nowhere. OpenAI said back in April that its GPT-5.5 model had been used to optimize the very Nvidia GPU cluster it runs on, with one tweak boosting token generation speed by more than 20%. The company has since shipped GPT-5.6, which it says is notably better at coding. Anthropic, meanwhile, built a Claude-powered agent to accelerate its own safety research, specifically to study weak-to-strong supervision, a technique researchers there think could help humans keep oversight of models that eventually outstrip them. In other words, the automation the letter frets about is already running in production labs, not sitting in a research paper somewhere.

And this letter lands in the middle of a busier-than-usual stretch of AI policy noise. Nvidia and other big tech firms just pushed back against a possible ban on open-source models. A group of economists put out their own statement calling for tougher AI regulation days before that. Demis Hassabis added his voice too, publishing an essay proposing a standards body that would build risk benchmarks and test models against them before release. None of these efforts agree on mechanism, but they're converging on the same basic instinct: the people closest to the technology increasingly think self-regulation alone isn't going to cut it, and they want government involved in setting the rules rather than leaving it purely to competitive pressure between labs.