StarCoder2-Instruct: Fully Transparent and Permissive Self-Alignment for Code Generation

Hugging Face · 2 years ago · 20

StarCoder2-15B-Instruct, a code generation model, was trained using only self-generated instruction-response pairs without relying on proprietary models like GPT-4 or human annotations. The model achieved a 72.6 HumanEval score, surpassing CodeLlama-70B-Instruct's 72.0 score, and outperformed variants trained on GPT-4 distilled data on the LiveCodeBench benchmark. The fully open-source pipeline and transparent methodology enable other researchers to build instruction-tuned code models without licensing restrictions from proprietary systems.