Why don’t machine learning research agents overfit?
Amazon Science 1 month ago 10
ML research agents repeatedly tune models using the same reusable benchmark-style validation loop, yet they still avoid strong overfitting, despite textbook predictions. The study reports that across eight datasets, 32-token prompts were enough for a fresh reproducer to match the explorer on most problems, and one language-modeling strategy compressed down to 16 tokens without losing held-out performance. The results suggest the tuned strategies are highly compressible, so generalization comes from information that can fit through a small “certificate” channel rather than from memorizing the validation set.