MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

OpenAI · 2 years ago · 52

Researchers created MLE-bench, a benchmark designed to evaluate how effectively AI agents can perform machine learning engineering tasks. The benchmark assesses agents across multiple dimensions of ML engineering work, measuring their capability to handle real-world engineering challenges. This enables more systematic evaluation of whether AI systems can assist with practical aspects of machine learning development beyond model training.