Anthropic researcher Jacob Coxon quits over self-improving AI risk

Anthropic researcher Jacob Coxon has walked out of the company and used the moment to deliver a blunt warning: the people building advanced AI think the clock is running down. His resignation post on X has already been viewed more than 100 million times, and in an interview with WIRED he said many colleagues share his view that the next year or two is “crunch time” for humanity.

Coxon isn’t talking about vague sci-fi dread. He points to recent security incidents, including the Hugging Face hack tied to OpenAI’s agent swarm, as evidence that models are already behaving in ways that used to sound absurd. He also says the pace of capabilities is rising fast in coding, hacking and math, while the industry keeps pushing ahead anyway.

His biggest concern is alignment: the ugly little term for making sure systems do what humans actually want. Coxon says no one has solved it, and that the current plan is to solve it at speed by using smarter models to do safety work on the next generation of models. He thinks that sounds optimistic at best and reckless at worst.

He also claims the danger isn’t some abstract lab puzzle. In his view, advanced systems could be used for biological threats or cyberweapons, and recursive self-improvement — AI building newer AI — is the point where things get really ugly. Once a system is smart enough to resist being turned off, he argues, the control problem stops being theoretical.

Coxon says Anthropic is more responsible than OpenAI, and that leadership there talks much more openly about strategy and risk. Still, he doesn’t trust any private company to run what he calls a mini Manhattan Project. Anthropic said it wants the industry to adopt a lawful, verifiable way to pace the release of powerful models. OpenAI didn’t comment.