Data Machina #250

Substack · 2 years ago · 34

Meta released Llama 3, an open-weight large language model family available in 8B and 70B parameter sizes with a 400B model still in training. The models were trained on 24,000 GPUs and over 15 trillion tokens, with an 8,192 token context window and 128,000 word vocabulary. Llama 3's open availability enables independent researchers, startups, and enterprises to deploy capable models locally and on cloud infrastructure at lower costs than proprietary alternatives.