Mixture-of-Agents Alignment: Harnessing the Collective Intelligence of Open-Source LLMs to Improve Post-Training

Together AI · 1 year ago

Researchers proposed Mixture-of-Agents Alignment (MoAA), a distillation method that distills multiple open-source large language models into single, efficient smaller models for improved performance. Llama-3.1-8B improved from 19.5 to 48.3 on Arena-Hard, and Gemma-2-9B improved from 42 to 55.6, while MoA synthetic data cost 15% less than GPT-4o to generate. The approach enables smaller models to achieve performance comparable to models 10 times their size and supports a self-improving development pipeline for open-source LLMs without relying on closed-source model supervision.