Rethinking how we measure AI intelligence

Google DeepMind · 11 months ago · 14

Google DeepMind and Kaggle launched Game Arena, an open-source benchmarking platform where AI models compete in strategic games to evaluate their capabilities. The platform will host its inaugural chess tournament on August 5 with eight frontier models competing, with final rankings determined by over one hundred matches between each model pair. Game Arena addresses limitations of current benchmarks by using games with clear winning conditions to measure reasoning and planning rather than memorization, with plans to expand to Go, poker, and video games.