Training 1000-layer networks without backpropagation
Sakana AI
PC-ALM introduces a local-learning training method that replaces backpropagation with layer-local dynamics for neural networks. It trains 1000-layer neural nets without backpropagation. The approach uses dual neurons (Lagrange multipliers) to propagate credit across depth, improving learning in deep narrow networks where predictive coding previously struggled.
Related stories
Sakana AI Researchers Introduce PC-ALM, a Layer-Local Alternative to Backpropagation That Trains 1000-Layer Networks
MarkTechPost · 2 weeks ago ·
26
Sakana AI’s Error Diffusion Trains Dale-Compliant Dual-Stream Networks, Reaching 96.7% MNIST and 61.7% CIFAR-10 Without Backpropagation
MarkTechPost · 2 months ago ·
32
DiffusionBlocks: Training Neural Networks One Block at a Time
Sakana AI ·
17