Does intelligence need a hard cap?
Platformer Casey Newton
Opinion — commentary, not a factual news event.
AI insiders are starting to talk about capping how smart models can get. That could turn a vague safety debate into a real limit on superhuman systems.
Based on reporting by Platformer, Casey Newton — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
A new idea is creeping into the AI safety debate: maybe large language models should hit a ceiling, not just a speed bump. At The Curve, a Berkeley conference packed with AI executives, nonprofit leaders, government officials, and a few journalists, multiple speakers argued that future systems may need hard limits on how intelligent they’re allowed to become.
That sounds abstract until you place it next to what’s making people nervous. The OpenAI-Hugging Face incident is still rattling the industry, and OpenAI and Anthropic have both recently published posts about recursive self-improvement — systems that can help research and train their own successors. If that path holds, releases could accelerate fast, and control could get a lot harder.
The people making these arguments weren’t offering a neat policy blueprint. But the options being discussed are serious: a speed limit on recursive self-improvement, restrictions on using frontier models for AI research, limits on compute and on how many copies of a system can run, or a ban on deploying models past a certain capability. Anthropic’s responsible scaling policy already gestures in that direction, and similar ideas have spread to other leading labs.
The catch is enforcement. Labs can’t do this alone, and neither can individual countries. The US government, for now, is not eager to help. There are narrower tools on the table — embedded evaluators, and maybe an antitrust waiver that lets labs coordinate on safety without turning into a cartel — but even supporters sound unconvinced that those measures are enough.
That gap matters because the industry and the government are not aligned on how close danger is. Lab leaders say catastrophe could arrive as soon as next year; Washington has bounced between wanting frontier-model licensing and cheering companies on to move faster. Meanwhile, the president is urging everyone to say “super intelligence,” as if the problem were mostly branding. It isn’t.
Opinion: The hard-cap idea is the first AI safety proposal in a while that sounds like it was written by people who expect the models to keep getting stronger, not by a committee of euphemisms. That alone makes it more serious than the usual “responsible innovation” wallpaper. The annoying part is that the adults in the room keep discovering they need a government that can still say no.
My take — AI-written commentary, not fact-checked reporting
The hard-cap idea is the first AI safety proposal in a while that sounds like it was written by people who expect the models to keep getting stronger, not by a committee of euphemisms. That alone makes it more serious than the usual “responsible innovation” wallpaper. The annoying part is that the adults in the room keep discovering they need a government that can still say no.
Read more about this at: Platformer
