Sakana AI Researchers Introduce PC-ALM, a Layer-Local Alternative to Backpropagation That Trains 1000-Layer Networks
MarkTechPost 1 month ago 26
Sakana AI researchers introduced Augmented Lagrangian Predictive Coding (PC-ALM), a layer-local learning method meant to replace backpropagation’s global credit assignment with predictive-coding updates. They report training 1000-layer residual MLPs on MNIST for 5 epochs while staying within about 2 percentage points of backprop accuracy. The method adds per-layer Lagrange multipliers and improves accuracy over standard predictive coding, with released MIT-licensed JAX reference code reproducing results on CPU.