Agent Evaluation Metric for multi-turn conversations
Amazon Web Services Surafel Lakew
The Agent Evaluation Metric (AEM) was introduced to evaluate multi-turn agents at the level of individual turns, since one early error can corrupt later turns that holistic scoring misses.
Why it matters
Multi-turn agents fail in ways single-turn evaluation misses: one early mistake corrupts every later turn. This post introduces the Agent Evaluation Metric (AEM), a decomposable, turn-level way to measure agent quality, applied to its first dimension, correctness, to pinpoint the turn that caused a failure and separate it from the turns that inherited it.