Humans in the loop miss a third of dangerous AI coding agent requests
The Register ● Covered by 30 sources
A browser game testing whether people can spot dangerous AI coding commands found players missed about a third of malicious requests. Turns out humans babysitting AI agents get tired and start rubber-stamping everything, which is exactly when bad stuff sneaks through.
Alex Wauters, a developer in Belgium, built a simple browser game this past May to answer a question that had been bugging him: are humans actually any good at catching dangerous commands from AI coding agents like Claude Code, or are we just pretending the 'approve or deny' prompt is a real safety net? After more than 40,000 playthroughs and 409,000 individual approve/deny decisions, the answer looks grim. Players let roughly one in three malicious requests through, and the game wasn't even trying to be sneaky about it.
The game gives players 60 seconds to approve or reject a stream of simulated permission requests, the same kind Claude Code fires off constantly during a coding session. Some are obviously nasty, like an rm -rf on a root directory, and those got caught most of the time. But scope violations, things like an agent asking to read AWS credential files or Kubernetes configs, slipped past 35 percent of the time. Typosquatted packages and curl requests to unfamiliar APIs fared almost as badly. The single worst offender was a command called npm run analyze, waved through nearly 65 percent of the time even though the game displayed exactly what that script would execute right above the approval button. People just weren't reading it.
Wauters says he built the thing after watching developers burn out on permission prompts and start reaching for the '--dangerously-skip-permissions' flag just to get through multi-hour agent runs without constant interruptions. That's the real tension here: babysit every command and you're exhausted and sloppy, or skip the babysitting entirely and hope nothing goes wrong. Anthropic's own telemetry backs this up, showing Claude Code users approve about 93 percent of prompts overall, with attentiveness dropping the more approvals someone sees in a row.
Anthropic has tried to soften the problem with something called auto mode, a model-based classifier that pre-screens requests and catches around 83 percent of what it labels overeager behavior before execution. That still leaves 17 percent getting through, and Anthropic itself frames auto mode as one layer of defense, not a replacement for sandboxing. Wauters agrees, and his advice is blunt: run these agents in sandboxes or cloud devcontainers, layer on tools like auto mode, and write hooks that force potentially risky actions to get context-checked before they're rubber-stamped. Human-in-the-loop, as a phrase, sounds reassuring. The data suggests it's mostly theater.
My take
Nobody should be shocked that tired humans clicking approve at high speed miss a third of the bad stuff — that's not a human failing, it's a design failing. Slapping a person in front of an AI agent and calling it a safety mechanism was always a way for companies to outsource liability rather than actually solve the problem, and this game just put a number on how badly that shortcut performs. The fix isn't more vigilance training, it's sandboxes, scoped permissions, and tooling that doesn't rely on someone reading a wall of logs correctly at 2am.
Read more about this at: The Register
