Treat prompt changes like code deploys

There's a weird double standard in how teams treat their LLM features. Nobody would push a code change without tests and a CI gate. But tweak a prompt, glance at one output, and it goes straight to production. The article makes a simple case: a prompt is a dependency just like any library, and changing it can shift behaviour with no warning at all.

The reason this fails so quietly is that bad prompts don't crash anything. A broken function throws an error or fails a test. A degraded prompt just drifts — tone changes, format slips, hallucinations creep in, accuracy dips — and the server still returns a clean 200 with text that reads perfectly plausible. There's nothing in the request/response cycle that flags the difference between a good output and a subtly worse one. The only fix is checking outputs against something before a customer does.

That's the job of an eval gate: a required checkpoint between editing a prompt and that prompt actually running in production. For it to work rather than become theater, a few things matter. The gate should fail closed, blocking by default if no eval has run yet. It should only trust the latest run, not some pass from three edits ago. It needs a read-only status check so people can inspect state without accidentally triggering a promotion. And instead of just saying