If coding is solved, what now?: Measuring the sloppiness of code
Earendil
LLMs can write code that works, but it can still be messy and bloated. A new look at “slop” says the real problem is keeping code human-readable.
Based on reporting by Earendil — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
LLMs are getting very good at producing code that passes checks. That’s not the same as producing code people actually want to live with. The annoying part is the stuff that still technically works: extra abstractions, duplicate logic, and decisions that make a project harder to reason about one week later than it was on day one.
The piece argues that this is where coding agents start to look less magical. They can generate huge amounts of code, but they do not reliably clean up the mess they create. In projects that add millions of lines of code per month, humans lose the ability to keep up. The industry, the author says, is full of confident talk about agents that “ship it,” while the actual quality signal remains fuzzy and mostly vibes-based.
That fuzziness is the core problem. Code is easy to verify: hidden tests can tell you whether it works. Sloppiness is different. Judging whether code is bloated, repetitive, or overly complex often comes down to human taste, and that makes it much harder to measure in a consistent way.
The author walks through the usual options and is not impressed. Asking an LLM to rate code quality directly is described as close to random. Pairwise judging with two solutions can even wobble when you rename the files. Human review is better for readability, but it does not scale to training runs or big benchmarks with multiple systems.
The most promising simple signal, at least in the author’s tests, is the change in lines of code. More code often means more slop. Two other metrics from the SlopCodeBench paper, verbosity and erosion, also showed a wide gap between established repositories and agent-generated code: on average, agent code was about twice as verbose and eroded. In the benchmark itself, where context is wiped between rounds to mimic real iterative use, even state-of-the-art models hit a 0% strict solve rate.
The broader point is blunt. The harder part of AI coding is no longer making something run. It is keeping it sane after the first draft, and that still seems to need human judgment more than the industry wants to admit.
My take — AI-written commentary, not fact-checked reporting
This is the part of AI coding the hype crowd keeps stepping around: working code is cheap, clean code is not. If a system can add endless lines but can’t keep its own mess in check, that’s not autonomy, that’s a very fast intern with a search budget. The real benchmark is whether humans still want to read it a month later.
Read more about this at: Earendil