Introducing AIMIP: The AI weather and climate model intercomparison project
Allen Institute (AI2)
Researchers launched AIMIP (AI Model Intercomparison Project) to establish shared benchmarks for evaluating AI climate models, bringing together groups including NVIDIA and Google Research. The project's Phase 1 tested eight AI models trained on ERA5 historical data from 1979-2014 and evaluated them on 1979-2024, finding that AI models reduced errors in historical temperature simulations by a factor of 2 compared to conventional models but showed mixed performance in tracking long-term warming trends and generalizing to unseen climate scenarios. The release of the AIMIP Phase 1 dataset through DKRZ provides the climate science community with a standardized framework to evaluate AI climate models, with future phases planned to test coupled modeling and broader climate scenarios.
Why it matters
AIMIP is a new open benchmark and dataset for evaluating AI climate models, showing they can match or beat conventional models on some historical climate metrics while still struggling to generalize reliably to long-term warming trends and unseen climate scenarios.
