FACTS Benchmark Suite: Systematically evaluating the factuality of large language models

Google DeepMind · 10 months ago · 43

Researchers released the FACTS Benchmark Suite, a new evaluation system comprising 3,513 examples designed to measure how accurately large language models answer factual questions across four categories: parametric knowledge, web search, multimodal reasoning, and grounded responses. Gemini 3 Pro achieved the highest overall FACTS Score of 68.8%, with error rates 55% lower than Gemini 2.5 Pro on search-based questions and 35% lower on parametric questions. The benchmark suite, now hosted on Kaggle, provides a public leaderboard to track LLM factuality improvements over time.