How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?

Apple Machine Learning Research · 2 weeks ago · 18

The paper tests autonomous MLE coding agents and finds that elaborate multi-agent and retrieval harnesses don’t improve results over a minimal-harness coding agent when using an equal time budget and the same frontier LLM backbone. It specifically reports that open-source state-of-the-art harnesses offer no advantages over a single-session minimal-harness baseline. As a result, the authors argue that the extra machinery layers are mostly redundant for current MLE benchmarks and that spending effort on hand-crafted harnesses around strong models gives poor returns.