Mistral 7B

Mistral AI · 3 years ago · 53 ● 2 sources

Mistral AI released Mistral 7B, a 7.3 billion parameter language model that outperforms Llama 2 13B across all benchmarks and approaches the performance of Llama 34B on many tasks. On reasoning, comprehension, and STEM tasks, Mistral 7B performs equivalently to a Llama 2 model more than 3 times its size. The model uses Grouped-query attention and Sliding Window Attention to enable faster inference and longer sequence handling, and is available under the Apache 2.0 license for unrestricted use across local, cloud, and open-source platforms.