What's so hard about continuous learning?

There's a persistent myth in AI circles that models could keep learning on the job, the same way a new hire gets sharper after six months at a company. The tech for that isn't the problem. Every LLM since GPT-3 already runs on the exact machinery needed: feed new data back through the training pipeline, update the weights, repeat. Labs do this internally all the time — it's a safe bet GPT-5 is quietly training on ChatGPT and Codex outputs right now, spitting out checkpoints that eventually surface as point releases. The reason you can't run a version of Codex that slowly masters your codebase isn't a missing algorithm. It's that nobody has figured out how to let that process run unsupervised without it going sideways.

And it does go sideways, reliably. Model training isn't a smooth curve where more data plus more compute equals more intelligence. It's messy, seed-dependent, and often lucky — train the same architecture on similar data a hundred times and you'll get a hundred meaningfully different models. Anthropic's Claude Sonnet 4 might reflect a real architectural breakthrough, or it might just be a good roll of the dice. Left alone, a self-updating model is far more likely to drift into a worse version of itself than to gradually become a domain expert. Someone has to be watching, ready to yank the model back from a bad training direction, which defeats the whole point of