DABStep: Data Agent Benchmark for Multi-step Reasoning

Hugging Face · 1 year ago · 10

Adyen and Hugging Face released DABstep, a benchmark containing over 450 real-world data analysis tasks designed to evaluate AI agents' multi-step reasoning capabilities. The most advanced reasoning-based agents achieved only 16% accuracy on the benchmark, revealing a significant gap between current models and practical data analysis requirements. The benchmark aims to drive progress in agentic workflows for data analysis by providing objective evaluation on tasks requiring both structured data processing and domain knowledge from financial operations.