DSGym: A holistic framework for evaluating and training data science agents

Together AI · 5 months ago

Researchers introduced DSGym, a unified framework for evaluating and training data science agents that integrates diverse benchmarks behind a single API and includes 90 bioinformatics tasks and 92 Kaggle competitions. A 4-billion parameter model trained on 2,000 generated examples from the framework achieved competitive performance with much larger open-source models on general analysis tasks while frontier models like Claude Sonnet 4.5 reached only 37% accuracy on multi-step reasoning tasks. The framework reveals that current models rely on memorization for general tasks and lack domain grounding for scientific problems, enabling more systematic development of reasoning-based data science agents.