Competition and Attraction Improve Model Fusion

Sakana AI ● 2 sources

Sakana AI presented a paper at GECCO'25 proposing M2N2, a method that uses evolutionary algorithms to merge AI models by automatically determining how to partition and combine them rather than requiring manual definition. The approach evolved an MNIST classifier from random networks and successfully merged a math specialist LLM with an agentic specialist LLM to handle both tasks better than existing methods. This evolutionary model fusion approach enables more flexible combination of specialized models while avoiding catastrophic forgetting seen in traditional fine-tuning.