What AI gets wrong and what failure teaches us
Microsoft 1 week ago 16
Jennifer Neville, a Microsoft Research partner research manager, explains how AI evaluation based on simple benchmarks misses failures that show up in realistic, user-driven multiturn and long-horizon tasks. She joined Microsoft in 2021, and emphasizes using practical evaluations and data checks to find where model behavior diverges from user needs. The result is a shift from standard benchmark testing toward evaluation designs that better predict and guide improvements in AI models for real workflows.