Evolving New Foundation Models: Unleashing the Power of Automating Model Development
Sakana AI ● Covered by 3 sources
Sakana AI published a paper in Nature Machine Intelligence describing Evolutionary Model Merge, a method that uses evolutionary algorithms to automatically combine existing open-source AI models into new foundation models optimized for specific tasks. The company created three Japanese foundation models (EvoLLM-JP, EvoVLM-JP, and EvoSDXL-JP) where their 7B-parameter Japanese math LLM matched the performance of previous 70B-parameter models without requiring gradient-based training. This approach enables automated foundation model development with minimal compute resources by systematically discovering effective model combinations that human experts might not intuitively discover.
Related stories
Developing Post-Training Technology to Adapt the Largest Open Foundation Model to Country-Specific Specifications
Sakana AI ·
28
ShinkaEvolve: Evolving New Algorithms with LLMs, Orders of Magnitude More Efficiently
Sakana AI ·
33
Sakana Fugu: A Multi-Agent Orchestration System as a Foundation Model
Sakana AI ·
25