Why I'm Still Bearish on LLMs After Navier-Stokes

A new post argues that the most impressive LLM demos have seduced people into pricing these systems as if full autonomy is already here. The reality, it says, is messier: current frontier models still need heavy oversight, guardrails, and a lot of handholding even on simple jobs. The headline wins — Navier-Stokes, FreeBSD RCEs, the Hugging Face incident — are treated as proof of maturity, but the author says they’re outliers, not evidence that models can be dropped into most knowledge-work roles and just get on with it.

The core complaint is structural. LLMs can be trained to handle tasks with clear success criteria, but small shifts inside those task families can trigger failure or reward hacking. Fixing that means rigorous specification by domain experts, and that’s expensive. Worse, specification is its own skill. In many fields, the overlap between the people who know the domain and the people who can write a robust spec is tiny.

The post uses hardware engineering as a telling comparison. A typical CPU project, it says anecdotally, can have about three times as many specification and validation engineers as design engineers, and sometimes a 5:1 ratio. Even then, the process isn’t simple write-once-and-forget. Specs evolve as implementation uncovers problems, and for some supposedly neat cases, like an executable ISA spec for a CPU family, verification against the high-level spec is still beyond current technology. So the work gets pushed down to lower-level specs that are costlier and more fragile.

The author’s one genuinely friendly case for agentic systems is pure mathematics. Navier-Stokes is described as the best possible setup: the theorem statement already is a rigorous spec, it has been examined for decades, and its Lean translation leans on battle-tested objects from mathlib. Even there, the system isn’t invulnerable; soundness bugs have let bogus proofs slip through before. That makes the point sharper, not weaker. If even that looks shaky, most knowledge work is in worse shape.

From there the argument turns commercial. The claim is that most firms will not be able to use fully autonomous LLMs, not because they’re too dumb to adopt them, but because the economics and verification burden don’t work. Only three groups look viable: firms that can tolerate cheap failure, firms with a few narrowly defined guarded tasks, and firms that already live with the cost of rigorous specification and validation, such as chip design and drug discovery. For the first two, the post says cheaper open models on cheap hardware, maybe even local hardware, make more sense than frontier systems. For the third, secrecy and cost both push in the same direction. The punchline is blunt: frontier labs may have sold a story about self-driving AI, but the real near-term world may be a datacenter full of brainlets, and that version is bottlenecked by humans.