Cheaper, Better, Faster, Stronger

Mistral AI · 2 years ago · 39

Mistral released Mixtral 8x22B, a sparse mixture-of-experts open model that activates only 39 billion parameters from a total of 141 billion while supporting five languages and native function calling. The model achieves faster inference than dense 70B models and outperforms other open-weight alternatives on standard benchmarks including MMLU, reasoning tasks, and mathematics problems. Mixtral 8x22B is distributed under Apache 2.0 license without restrictions, making it available for anyone to use, fine-tune, and deploy in production applications.