2023, year of open LLMs

Hugging Face · 2 years ago · 44

Meta released LLaMA in February 2023, followed by a wave of smaller, open-source language models including Mistral, Qwen, and Falcon released throughout the year. These models ranged from 3 billion to 70 billion parameters, with the best performers trained on 1 to 1.5 trillion tokens of data. The shift toward smaller models trained on more data made high-quality language models accessible to researchers and organizations with limited computing budgets.