Training 1000-layer networks without backpropagation
Sakana AI 19
PC-ALM introduces a local-learning training method that replaces backpropagation with layer-local dynamics for neural networks. It trains 1000-layer neural nets without backpropagation. The approach uses dual neurons (Lagrange multipliers) to propagate credit across depth, improving learning in deep narrow networks where predictive coding previously struggled.