Qwen2.5-Math: The world's leading open-sourced mathematical LLMs
GitHub Pages
Qwen dropped a new open-source math LLM family, Qwen2.5-Math, and it's beating GPT-4o and Gemini's math specialist on tough benchmarks. The wild part: a 7B model now matches its old 72B version.
Based on reporting by GitHub Pages — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Alibaba's Qwen team just pushed out Qwen2.5-Math, and the numbers are the kind that make you double-check the source. The flagship 72B-Instruct model hits 92.9 on MATH when paired with tool-integrated reasoning and a reward model — beating GPT-4o and Google's math-specialized Gemini 1.5 Pro on the same test. That's not a marginal bump. It's the kind of gap that used to separate open and closed models by a full generation.
What's more interesting than the raw scores is how they got there. The team didn't just scale up more data — they used Qwen2-Math-72B-Instruct to synthesize fresh training data, then swapped in Qwen2.5's base models for better language and code understanding, and rebuilt the training corpus from 700 billion tokens to over a trillion. They also trained a dedicated 72B reward model to grade candidate solutions and used it both for rejection sampling during fine-tuning and for reinforcement learning with GRPO afterward. It's a pipeline that treats math ability less like a single training run and more like a refinery process, passing outputs back through the system to polish them.
The practical upshot is that model size stops mattering as much as it used to. Qwen2.5-Math-1.5B-Instruct, a model small enough to run on a laptop, now beats most 70B-class math models from a year ago. The 7B version actually surpasses the old Qwen2-Math-72B-Instruct on MATH, scoring 83.6 with chain-of-thought and 85.3 with tool-integrated reasoning, which lets the model write and execute Python instead of just talking through the steps. On AIME 2024, arguably the nastiest benchmark here, Claude 3 Opus, GPT-4 Turbo and Gemini 1.5 Pro each solve only 1 or 2 of 30 problems. Qwen2.5-Math-72B-Instruct solves 12 with TIR, and the 7B model, boosted by the reward model, gets up to 21.
None of this is a general-purpose chatbot upgrade — Qwen is explicit that this series is narrow, tuned for English and Chinese math problems and not much else. But narrow and open isn't a bad combination. If you need a model that can actually verify a derivative or solve a system of equations without hallucinating a wrong root, this is now sitting near the top of the pile, code and weights included, no API key required.
My take — AI-written commentary, not fact-checked reporting
I'll say what a lot of people are dancing around: a 1.5B open model outperforming last year's 70B giants is the actual headline here, not the leaderboard flex against GPT-4o. Efficiency gains like this are exactly why betting everything on ever-bigger closed models feels increasingly like a rich person's hobby rather than a technical necessity — and it's happening in the open, from a Chinese lab, while Western AI policy debates are still stuck arguing about whether open weights are dangerous.
Read more about this at: GitHub Pages