StarCoder2 and The Stack v2

Hugging Face Blog · 2 years ago

BigCode released StarCoder2, a family of open-source code language models in three sizes trained on The Stack v2, a new high-quality code dataset. The flagship StarCoder2-15B model was trained on over 4 trillion tokens across 600+ programming languages, while the smaller 3B variant matches the performance of the original StarCoder1-15B model. All models, datasets, and training code are publicly available, enabling developers to build and fine-tune code applications without proprietary dependencies.