Evolving New Foundation Models: Unleashing the Power of Automating Model Development

Sakana AI ● 3 sources

Sakana AI published a paper in Nature Machine Intelligence describing Evolutionary Model Merge, a method that uses evolutionary algorithms to automatically combine existing open-source AI models into new foundation models optimized for specific tasks. The company created three Japanese foundation models (EvoLLM-JP, EvoVLM-JP, and EvoSDXL-JP) where their 7B-parameter Japanese math LLM matched the performance of previous 70B-parameter models without requiring gradient-based training. This approach enables automated foundation model development with minimal compute resources by systematically discovering effective model combinations that human experts might not intuitively discover.