TLDRocket
Sign in
Latest Quoting The New York Times — Simon Willison’s Weblog Why AlphaFold Didn't Solve Protein Folding — Pushmeet Kohli, Google De... — Latent Space Anthropic can’t reliably control its AI agents. It’s cutting off its i... — TechCrunch IBM connects enterprise AI orchestration to production readiness ahead... — SiliconANGLE Doctor Evidence Search Tool “Evidence Finder” Adds Sakana Namazu — Sakana AI Ukraine’s drones knock out AI data center belonging to "Russia’s Googl... — Ars Technica The maker of non-text AI model Jev valued at $7.5B just weeks after la... — TechCrunch I expect rapid progress but not towards general superintelligence — Interconnects

The AI intelligence platform

Every AI story that matters — and the intelligence behind it.

TLDRocket reads all relevant sources, removes duplicate coverage, and publishes a short neutral summary of every story, linking back to the original. Free, no spam, unsubscribe anytime.

Add to Slack

Every story also updates live profiles event timelines weekly rankings the AI Market Index

Friday, 2 February 2024

NPHardEval Leaderboard: Unveiling the Reasoning Abilities of Large Language Models through Complexity Classes and Dynamic Updates

Hugging Face 2 years ago 32

Researchers from the University of Michigan and Rutgers University launched NPHardEval, a new leaderboard that evaluates large language models' reasoning abilities using 900 algorithmic questions spanning computational complexity classes. The benchmark contains 100 questions each for 9 different algorithms across 10 difficulty levels, with monthly updates to prevent models from overfitting to static test sets. The evaluation framework enables more reliable measurement of logical reasoning by using automatically generated and verified questions grounded in computational complexity theory, with GPT-4 Turbo currently achieving the highest performance among tested models.

Read the full story
Benchmarking

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.