An LLM-as-Judge Won't Save The Product—Fixing Your Process Will

Eugene Yan's latest post is basically an intervention for teams who think they can buy their way out of a broken AI product with one more eval tool. His argument: evals aren't gadgets you install, they're a discipline you practice, and most teams skip the discipline part.

The core of his pitch is that good evals are just the scientific method wearing a hoodie. You look at the data first — actual inputs, actual outputs, actual user behavior — before you touch a metric. Then you annotate a representative sample, ideally a 50:50 split of passes and fails, so your dataset actually reflects where things break. From there you form a hypothesis about why something's failing (bad retrieval, conflicting instructions, whatever) and run an experiment with a defined success condition and, ideally, a baseline to compare against. If it works, you ship it. If it doesn't, you go back and refine the hypothesis. Yan calls this the flywheel, and he means it literally: skip a step and the wheel stops turning.

He also draws a direct line to eval-driven development, his riff on test-driven development. Instead of writing code then hoping it works, you define your success criteria — your evals — before building the feature. Then every prompt tweak or retrieval change gets measured against that baseline instead of judged by vibes. Machine learning teams have quietly done this for years with validation and test sets; Yan's just repackaging it for the LLM era with a catchier name.

The part that feels most pointed is his takedown of LLM-as-judge as a silver bullet. Automated evaluators are useful for scaling monitoring, sure, but they only work if they're calibrated against real human annotations, and even then they need continuous human oversight — sampling outputs, reading customer feedback, catching what the automated system misses. Yan's blunt about it: neglect isn't something a judge model can compensate for. No tool absolves you of actually looking at your data.

What makes this piece land is how unglamorous the advice is. Everyone wants an AI silver bullet; Yan's telling teams the boring truth that the fix is process discipline, not procurement. Building AI products, he writes, still takes elbow grease — a phrase that undercuts a lot of the magic-wand marketing floating around this space right now.