TLDRocket
Sign in
Latest 20% of Americans are already using AI for financial advice — another 7... — Fortune Auto mode is now the default in Claude Code for Pro, Max, and Team pla... — Simon Willison’s Weblog AI labs shouldn't be allowed to grade their own homework — Fortune AI is changing work faster than the data can keep up — Fortune Planned Amazon data center could become the biggest climate polluter i... — TechCrunch Meet Shepherd: An Open-Source Python Substrate That Lets Meta-Agents F... — MarkTechPost OpenAI acquires presentation startup NextSlide — TechCrunch Pokee AI Releases Pokee-Isaac 28B: A 10M-Token Context Agentic Model B... — MarkTechPost

The AI intelligence platform

Every AI story that matters and the intelligence behind it.

TLDRocket reads all relevant sources, removes duplicate coverage, and publishes a short neutral summary of every story, linking back to the original. Free, no spam, unsubscribe anytime.

Add to Slack

Every story also updates live profiles event timelines weekly rankings the AI Market Index

Friday, 14 February 2025

Fixing Open LLM Leaderboard with Math-Verify

Hugging Face 1 year ago 14

The Open LLM Leaderboard re-evaluated all 3,751 submitted models using Math-Verify, a new evaluation tool that fixes parsing and comparison failures in mathematical reasoning tasks. Models improved by an average of 4.66 points, with algebra-related subsets showing gains of 6.93 to 8.27 points, while some models improved by nearly 90 points on specific subsets. The ranking reshuffling elevated Qwen and DeepSeek models substantially—Qwen scores more than doubled and DeepSeek scores nearly tripled—while Nvidia's AceMath models now dominate the MATH-Hard leaderboard top positions.

Welcome Fireworks.ai on the Hub 🎆

Hugging Face 1 year ago 24

Fireworks.ai is now available as a supported inference provider on Hugging Face Hub, allowing users to run serverless inference on models including DeepSeek-R1, DeepSeek-V3, Mistral-Small, and Llama-3.2-90B. Users can access Fireworks inference through the Hub's website UI, Python and JavaScript SDKs, or HTTP calls, with billing passed through directly at standard Fireworks API rates and no markup from Hugging Face. PRO subscribers receive $2 in monthly inference credits that work across all providers on the platform.

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.