Your Agent Aced the Task. Will It Do It Again?
Hugging Face ● Covered by 2 sources
Your AI agent can ace a task once and still flub it the next time. Hugging Face says the real bug is inconsistency, not just low accuracy.
Based on reporting by Hugging Face — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
A new Hugging Face blog post argues that agent benchmarks have been flattering the wrong number. A ReAct agent using GPT-4.1 on AppWorld test_normal posts a Mean@5 of 77.4%, which looks solid. But only 53.0% of tasks succeed in all five runs. That 24.4-point gap is the part users actually feel when a system behaves one way in rehearsal and another way in production.
The post makes a clean distinction that most scoreboards blur together. Mean@k is the familiar average. Pass^k is harsher: the task only counts if every run succeeds. The site’s point is simple enough to be annoying — an agent can be capable and still be unreliable. And if the task is something like reconciling money or checking a contract, that’s not a cute benchmark quirk. It’s a problem.
Hugging Face says the variability comes from the model’s own decision points, especially the flat ones. Some token choices are sharp and stable; others are near-ties that can flip because of small platform-side nudges, even at temperature 0.0. Since an agent makes dozens of these decisions in a row, a little wobble can turn into a very different trajectory. Greedy decoding and fixed seeds do not solve that, because they govern how a distribution is sampled, not how decisive the distribution is.
To catch those wobbly steps, the team built a Consistency Analyzer. It takes one recorded trajectory, resamples each decision point with a single extra model call, and flags the places where the output is most likely to flip. No logits. No internals. No replaying the task end-to-end. Those flagged steps are then turned into consistency guidelines inside ALTK-Evolve, so the agent gets nudged away from the fragile choices it was already making.
The result, on AppWorld test_normal with 168 tasks, is a noticeable reduction in the gap. Pass^5 rises from 53.0% to 69.0%, while Mean@5 also edges up from 77.4% to 81.0%. The same pattern shows up on related tasks and even on gpt-oss-120b, though from a much lower base. The blog’s broader message is blunt: if you ship an agent and only report the average, you may be measuring luck with better branding.
My take — AI-written commentary, not fact-checked reporting
This is the part of agent eval people keep trying to sand off with bigger models and prettier charts. Bad idea. If a system only works when the universe is feeling cooperative, it’s not production-ready, it’s a lottery ticket with API keys. The annoying truth is that consistency deserves its own metric, and yes, that means the leaderboard gets less flattering.
Read more about this at: Hugging Face
