Sakana AI’s Error Diffusion Trains Dale-Compliant Dual-Stream Networks, Reaching 96.7% MNIST and 61.7% CIFAR-10 Without Backpropagation

MarkTechPost · 6 hours ago

Sakana AI developed Error Diffusion, a local learning rule that trains neural networks compliant with Dale's principle (separate excitatory and inhibitory neurons with non-negative weights) without using backpropagation or weight transport. The method achieved 96.7% accuracy on MNIST and 61.7% on CIFAR-10, with three key innovations including layer-specific sigmoid widths and batch-centered error routing. The approach represents the first demonstration of Error Diffusion on convolutional networks and reinforcement learning tasks, though performance lags behind standard backpropagation methods.