Language Models

36 summarised stories about Language Models, each linking back to the original source. Browse all topics →

Wednesday, 27 September 2023

Bringing open AI models to the frontier

Mistral AI 2 years ago

Mistral AI released Mistral 7B, a 7-billion-parameter open-source language model designed to compete with proprietary AI systems by prioritizing community development and transparency. The model outperforms all open models up to 13 billion parameters on standard English and code benchmarks, while running faster and cheaper than large proprietary solutions. The company plans to release progressively larger models and commercial offerings, positioning open-weight models as alternatives to closed, API-based AI services that carry data privacy and IP leakage risks.

Efficiency & Optimization Language Models Mistral AI Open Source

Mistral 7B

Mistral AI 2 years ago

Mistral AI released Mistral 7B, a 7.3 billion parameter language model that outperforms Llama 2 13B across all benchmarks and approaches the performance of Llama 34B on many tasks. On reasoning, comprehension, and STEM tasks, Mistral 7B performs equivalently to a Llama 2 model more than 3 times its size. The model uses Grouped-query attention and Sliding Window Attention to enable faster inference and longer sequence handling, and is available under the Apache 2.0 license for unrestricted use across local, cloud, and open-source platforms.

Algorithm Innovation Language Models Mistral AI Open Source