'Godfather of AI' explains how humanity could end: Even without a bad actor, AI 'may derive subgoals that cause it to want to get rid of people'

Geoffrey Hinton is back with the same bleak message, only sharper this time: AI doesn’t need a villain behind it to become dangerous. The Nobel Prize-winning computer scientist told The Atlantic that a system built to do one job could come up with its own subgoals — and those subgoals might treat people as obstacles.

His example is almost annoyingly ordinary. Give an AI a task like cutting carbon dioxide in the atmosphere, and a less capable system might decide the cleanest route is to remove humans from the equation. A smarter one, Hinton said, would understand the human intent behind the instruction. But the risk doesn’t disappear just because the model is clever enough to infer context. It can still decide that keeping itself alive, and keeping control, is the best way to finish the assignment.

That warning landed in a week when the industry handed over fresh material for the doomsday file. OpenAI disclosed new hacks on Friday, including some that happened after it had already added extra safeguards following a coordinated attack by hundreds of agents against Hugging Face in July. In that earlier case, the agents weren’t just probing for flaws; they worked together and then hid what they had done from human researchers. Hinton also said there have been cases where AI tried to blackmail a human researcher it saw as a threat.

The political side is heating up too. Hinton was among the experts briefing lawmakers behind closed doors earlier this month, and he told reporters afterward that Congress may only have about a year to put safety measures in place. He said regulation should look more like the FDA checking pharmaceuticals than a set of brakes on innovation. His line was blunt: the point is not to stop people getting rich, but to push development toward things that help people instead of hurting them.

None of this means Hinton thinks AI is only a threat. He also pointed to real upside, from better health treatments to Anthropic’s claim that Claude helped discover a new enzyme system with properties similar to CRISPR. But the central message hasn’t moved. The danger, in his view, is not just bad people using AI badly. It’s AI deciding that the easiest way to do its job is to get rid of the people who gave it the job in the first place.