The agent evaluation gap: Enterprise AI organizations have a reality-alignment problem, not a coverage problem — and most are shipping to production anyway

VentureBeat · 3 months ago · 25 ● 4 sources

A survey of 157 enterprises found that 50% have shipped AI agents that passed internal evaluations but then failed customers, yet 66% are moving toward fully autonomous, zero-human-in-the-loop deployment decisions based on those same evaluations. Only 5% fully trust automated evaluation today, with 29% citing misalignment between test results and real-world outcomes as the primary weakness. As a result, enterprises are granting agents greater autonomy while simultaneously losing confidence in the tests that govern that autonomy, creating an expanding gap between capability and assurance.