CO₂ Emissions and Models Performance: Insights from the Open LLM Leaderboard

Hugging Face Blog · 1 year ago

The Open LLM Leaderboard integrated carbon emission estimates into its evaluation of over 3,000 language models since June 2024 to measure the environmental cost of model inference. Community fine-tuned models consumed less than 5 kg of CO₂ while achieving average scores around 35 on models below 10 billion parameters, and surprisingly showed better energy efficiency than the official models they were derived from. This transparency enables model creators to optimize for both performance and environmental responsibility, revealing that instruction-tuned base models often produce verbose outputs that inflate emissions compared to properly fine-tuned versions.