New Paper: Towards a science of AI agent reliability
AI Snake Oil Sayash Kapoor
Researchers released a paper introducing a framework for measuring AI agent reliability across 12 dimensions (consistency, robustness, calibration, and safety), evaluating 14 models from OpenAI, Google, and Anthropic over 18 months using two benchmarks with 500 total runs. While accuracy improved substantially over this period, reliability gains were modest, with consistency scores ranging from 30% to 75% and agents performing poorly at recognizing when they are wrong. The findings suggest that deployers should distinguish between automation and augmentation use cases, and that researchers should measure and optimize for reliability as a separate dimension from accuracy rather than relying on single-run benchmark scores.
Why it matters
Quantifying the capability-reliability gap
