AGI is not a milestone

Every few months someone at a big lab declares that AGI is basically here, or nearly here, and the internet spends a week arguing about definitions. The release of OpenAI's o3 kicked off another round of this, with commentators like Tyler Cowen and Ethan Mollick essentially saying: I know it when I see it, and I see it. A new essay from the AI Snake Oil team takes a different tack entirely. Their argument isn't that o3 fails to qualify as AGI. It's that the entire concept of AGI-as-milestone is broken, borrowed from a mental model that doesn't apply here at all.

That model is the Manhattan Project. A nuclear detonation is unmistakable and its consequences are immediate — a war ends, a new geopolitical order begins within weeks. AGI, the authors argue, has neither property. There's no clean capability threshold that flips a switch, and even a genuinely superhuman general-purpose system wouldn't reorder the economy or world politics overnight. They walk through a thought experiment: take o3's architecture, make it much better at finding information and running code, but leave it just as unable to learn from new experience on the fly. That system could plausibly satisfy several existing AGI definitions — it might crush humans at chess by simply invoking a chess engine — while still failing badly at ordinary real-world tasks. Which, they say, is exactly the point: the label ends up saying more about how loosely we define AGI than about what the system can actually do.

The piece leans hard on history. Electricity, computing, the internet — none of these transformed the economy the moment the core technology existed. It took decades of complementary work: new products, retrained workers, new organizational habits, new laws. The Industrial Revolution's real miracle wasn't a growth spike, it was decades of sustained growth under 3% a year. AI, they argue, will follow the same slow diffusion curve no matter how capable the underlying model gets, because the bottlenecks were never really about model capability in the first place.

The geopolitics argument follows similar logic. The US-China AI race is often framed as a sprint to AGI with a winner-take-all prize at the end, echoing nuclear deterrence logic. But AI know-how leaks fast — it lives in hundreds of thousands of private-sector engineers, not classified government labs — so lasting technical leads are hard to sustain. The DeepSeek moment earlier this year, they note, caught people off guard precisely because they underestimated how quickly capability proliferates. The authors think China's real disadvantage isn't model quality, which trails the US by maybe six to twelve months, but weaker digitization, cloud infrastructure, and workforce readiness — the diffusion machinery, not the invention machinery.

They close by untangling capability from power in the safety debate, arguing that fears of an AI-driven loss of control smuggle in an assumption that raw capability equals the power to act on the world unsupervised. Separate those two things, they say, and the idea of a single point-of-no-return moment for humanity mostly dissolves. Whatever you think about o3 specifically, their broader claim is worth sitting with: AGI, as a term, was built to describe an event. What's actually unfolding is a process, and processes don't make good headlines.