Introducing Nested Learning: A new ML paradigm for continual learning

Google Research · 8 months ago

Researchers introduced Nested Learning, a machine learning paradigm that unifies model architecture and optimization algorithms as interconnected multi-level learning problems to address catastrophic forgetting in continual learning. A self-modifying architecture called Hope, presented at NeurIPS 2025, demonstrated lower perplexity and superior long-context memory management compared to standard transformers and recurrent models on language modeling and reasoning tasks. This approach enables models to incorporate multiple update frequencies and memory levels, potentially enabling continuous knowledge acquisition without losing previous capabilities.