NPHardEval Leaderboard: Unveiling the Reasoning Abilities of Large Language Models through Complexity Classes and Dynamic Updates

Hugging Face · 2 years ago · 32

Researchers from the University of Michigan and Rutgers University launched NPHardEval, a new leaderboard that evaluates large language models' reasoning abilities using 900 algorithmic questions spanning computational complexity classes. The benchmark contains 100 questions each for 9 different algorithms across 10 difficulty levels, with monthly updates to prevent models from overfitting to static test sets. The evaluation framework enables more reliable measurement of logical reasoning by using automatically generated and verified questions grounded in computational complexity theory, with GPT-4 Turbo currently achieving the highest performance among tested models.