Sakana AI’s Error Diffusion Trains Dale-Compliant Dual-Stream Networks, Reaching 96.7% MNIST and 61.7% CIFAR-10 Without Backpropagation
MarkTechPost Asif Razzaq ● Covered by 2 sources
Sakana AI developed Error Diffusion, a local learning rule that trains neural networks compliant with Dale's principle (separate excitatory and inhibitory neurons with non-negative weights) without using backpropagation or weight transport. The method achieved 96.7% accuracy on MNIST and 61.7% on CIFAR-10, with three key innovations including layer-specific sigmoid widths and batch-centered error routing. The approach represents the first demonstration of Error Diffusion on convolutional networks and reinforcement learning tasks, though performance lags behind standard backpropagation methods.
Why it matters
Backpropagation relies on weight transport, which biological circuits likely cannot implement. Sakana AI's Error Diffusion sidesteps that constraint, training dual-stream excitatory/inhibitory networks that obey Dale's principle. This piece breaks down how modulo error routing scales the rule from MNIST to CIFAR-10 and reinforcement learning, and what its task-dependent ablations reveal. The post Sakana AI’s Error Diffusion Trains Dale-Compliant Dual-Stream Networks, Reaching 96.7% MNIST and 61.7% CIFAR-10 Without Backpropagation appeared first on MarkTechPost.
