DSGym: A holistic framework for evaluating and training data science agents
Together AI
Researchers introduced DSGym, a unified framework for evaluating and training data science agents that integrates diverse benchmarks behind a single API and includes 90 bioinformatics tasks and 92 Kaggle competitions. A 4-billion parameter model trained on 2,000 generated examples from the framework achieved competitive performance with much larger open-source models on general analysis tasks while frontier models like Claude Sonnet 4.5 reached only 37% accuracy on multi-step reasoning tasks. The framework reveals that current models rely on memorization for general tasks and lack domain grounding for scientific problems, enabling more systematic development of reasoning-based data science agents.
Why it matters
Introducing DSGym—a holisti evaluation and training framework for LLM-based data science agents. Features 90+ bioinformatics tasks, 92 Kaggle competitions, and synthetic trajectory generation. Our 4B model achieves state-of-the-art performance among open-source models through exe