Decoherence

This essay puts a name on a familiar kind of mess: decoherence. It’s what happens when a system accumulates lots of locally sensible models that don’t quite agree anymore, so every change spends more time translating between them and dodging side effects than doing useful work.

The core claim is simple and a little grim. Decoherence grows with the pace of change, and once a system gets messy enough, each new change makes the next one harder still. That’s the loop. Not a linear tax, but something that can run away.

The author uses plumbing to make it concrete. You can replace pipes all at once, floor by floor, or one failure at a time. The last option feels cheapest in the moment, but it leaves behind a patchwork of old and new parts, and that patchwork becomes harder to understand, harder to extend, and easier to break when someone asks for more pressure.

Software is the sharper version of the same problem because the maze is invisible. A developer can keep hacking away at it, but each patch can force more context into the next patch, and if management wants it faster, quality drops and complexity rises. Start a second transition before the first one is done, and the combinations multiply fast.

The essay’s punchline is that AI doesn’t save us from this. It may actually accelerate decoherence by producing more changes than humans can sanely absorb. So the feared intelligence explosion gets replaced by something duller and nastier: systems that become too tangled to improve safely, even with more agents, more compute, and more optimism.