Sakana AI Introduces DiffusionBlocks, a Block-wise Neural Network Training Method Reducing Memory Requirements
Research publication Provisional 75% confidence first seen
Sakana AI developed DiffusionBlocks, a training method that splits neural networks into independently trained blocks by treating the forward pass as a diffusion model denoising process. The approach significantly reduces memory requirements from linear growth with network depth to memory for a single block while maintaining performance on vision and language models, and was accepted at ICLR 2026.