Why don’t machine learning research agents overfit?
Amazon Science
ML research agents repeatedly tune models using the same reusable benchmark-style validation loop, yet they still avoid strong overfitting, despite textbook predictions. The study reports that across eight datasets, 32-token prompts were enough for a fresh reproducer to match the explorer on most problems, and one language-modeling strategy compressed down to 16 tokens without losing held-out performance. The results suggest the tuned strategies are highly compressible, so generalization comes from information that can fit through a small “certificate” channel rather than from memorizing the validation set.
Why it matters
New research indicates that AI agents learn compressible models of data, which don’t have enough space to enable memorization.