AMD Releases Instella-MoE-16B-A3B: A Fully Open Mixture-of-Experts LLM With 2.8B Active Parameters Trained On Instinct GPUs
MarkTechPost Asif Razzaq ● Covered by 33 sources
AMD released Instella-MoE-16B-A3B, a fully open Mixture-of-Experts language model trained on Instinct MI300X and MI325X GPUs with full transparency on weights, data, and training configurations. The model uses 16 billion total parameters but activates only 2.8 billion per token, achieving a base benchmark score of 76.7 and post-training score of 73.22, both leading fully open competitors. The weights are restricted to research use under ResearchRAIL license, but the MIT-licensed training code is freely available for academic labs and enterprise teams to reproduce the recipe and deploy expert-parallel serving with 39.2% reduction in time to first token.
Why it matters
AMD released Instella-MoE-16B-A3B, a fully open Mixture-of-Experts language model trained from scratch on Instinct MI300X and MI325X GPUs. It holds 16B total parameters but activates only 2.8B per token, using Gated MLA and FarSkip-Collective. AMD published weights from every training stage, plus data mixtures, configs, and inference code. The post AMD Releases Instella-MoE-16B-A3B: A Fully Open Mixture-of-Experts LLM With 2.8B Active Parameters Trained On Instinct GPUs appeared first on MarkTechPost.
