Mistral NeMo

Mistral AI · 1 year ago

Mistral released NeMo, a 12-billion-parameter model developed with NVIDIA that supports a 128,000-token context window and is available under the Apache 2.0 license. The model uses a new tokenizer called Tekken that compresses source code 30% more efficiently than previous Mistral tokenizers and is 2-3 times more efficient for Korean and Arabic. NeMo can serve as a drop-in replacement for Mistral 7B systems and supports multilingual applications across 11 languages including English, Chinese, Arabic, and Hindi.