Qwen2.5-LLM: Extending the boundary of LLMs

GitHub Pages · 2 years ago · 24 ● 3 sources

Alibaba released Qwen2.5, a series of seven open-source language models ranging from 0.5B to 72B parameters, with new mid-size models at 14B and 32B designed for production use. The pre-training dataset expanded from 7 trillion to 18 trillion tokens, and Qwen2.5-72B achieved an MMLU score of 86.1 compared to Qwen2-72B's 84.2, while Qwen2.5-32B outperformed the larger Qwen2-72B in various benchmarks. The models show significant improvements in coding, mathematics, and long-context generation, with Qwen2.5-72B-Instruct reaching an Arena-Hard score of 81.2 and LiveCodeBench score of 55.5.