AI’s best coding agent fails 60% of the time — and the data backs it up
The New Stack 1 month ago 27 ● 2 sources
Claude Fable 5.1 topped the Real-SWE coding benchmark while failing on most attempts in private, company-code-style tests. The agent scored 38.8%, meaning it missed more than 60% of tasks (6 out of 10 tries). Real-SWE’s shift to private code and full tool setups makes coding-agent results drop quickly in unfamiliar production-like codebases and shows no model was consistently reliable.