Evaluating RAG with LLM as a Judge

Mistral AI · 1 year ago

Mistral published guidance on evaluating retrieval-augmented generation systems using LLMs as judges to assess whether generated answers are accurate and grounded in retrieved data. The RAG Triad framework evaluates three metrics: context relevance, groundedness, and answer relevance, each scored on a numerical or categorical scale. Mistral's structured outputs feature enables consistent, machine-readable evaluation formats to build more reliable automated assessment systems for LLM applications.