Granite 4.1 LLMs: How They’re Built
Hugging Face
IBM detailed exactly how it built Granite 4.1, a family of 3B/8B/30B dense models trained on 15T tokens. Their 8B model now beats IBM's own bigger 32B MoE model, using a much simpler design.
Based on reporting by Hugging Face — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
IBM just pulled back the curtain on Granite 4.1, and the headline isn't the model itself so much as the process behind it. These are dense, decoder-only transformers in three sizes — 3B, 8B, and 30B — trained from scratch on roughly 15 trillion tokens. Nothing exotic architecturally: Grouped Query Attention, RoPE, SwiGLU, RMSNorm. The interesting part is what IBM did with the data.
The pre-training pipeline runs five distinct phases, each shifting the mixture toward higher quality. Phase 1 burns through 10 trillion tokens of mostly CommonCrawl and code to build general fluency. Phase 2 dumps in far more math and code — a five-fold jump in math data alone — to sharpen reasoning. Phases 3 and 4 taper the learning rate toward zero while blending in chain-of-thought traces and instruction data, essentially annealing the model into shape. Phase 5 then stretches context from 4K all the way to 512K tokens through staged extension, with model merges after each stage so long-context ability doesn't wreck short-context performance. On the RULER long-context benchmark, the 30B model holds 76.7 at 128K tokens, which is a solid result for a dense model that size.
After pre-training comes supervised fine-tuning on about 4.1 million curated samples, filtered through an LLM-as-Judge system that scores responses on six dimensions — correctness, completeness, naturalness, calibration, and so on — and auto-rejects anything with hallucinations or bad computation, no matter how it scores otherwise. Then IBM layers on a four-stage reinforcement learning pipeline using on-policy GRPO with DAPO loss: multi-domain RL first to avoid catastrophic forgetting, then an RLHF pass that alone lifted AlpacaEval scores by almost 19 points on average, a short identity-calibration stage, and finally a dedicated math RL stage to claw back the math performance that RLHF tends to erode.
The payoff is the part IBM clearly wants people to notice: the 8B Granite 4.1 instruct model now matches or beats Granite 4.0-H-Small, a 32B-parameter mixture-of-experts model with 9B active parameters. A smaller, architecturally simpler dense model outperforming last generation's bigger MoE setup is not a small claim, and it says more about IBM's data curation and RL recipe than about raw scale. All three sizes ship under Apache 2.0, so anyone can go verify the numbers themselves.
My take — AI-written commentary, not fact-checked reporting
This is the most useful kind of AI paper right now — not another benchmark flex, but an actual admission that data curation and multi-stage RL matter more than parameter count. IBM proving an 8B dense model can outrun its own 32B MoE predecessor should worry anyone betting their roadmap purely on scale. And releasing the whole thing under Apache 2.0 while writing up the recipe in this much detail is exactly the kind of openness closed labs won't give you, which is why Granite keeps being underrated.
Read more about this at: Hugging Face
