TLDRocket
Sign in
Latest The Agent Said It Was Done. The Database Disagreed. — Hugging Face September sponsors-only newsletter — Simon Willison’s Weblog 9 insights from ‘Private Tech Trailblazers’: Vertical AI becomes the g... — SiliconANGLE OpenAI safety employee resigns, claiming the company’s ‘culture is bro... — TechCrunch Aleph Alpha’s Sovereign A.I. Model Kolibri Is No Match for the Open-We... — Trending Topics AI is speeding up exploits. Vulnerability spreadsheets can’t keep up. — The New Stack All the AI agents that can live in your text messages — TechCrunch A.I. Boom Is ‘the Only Reason We’re Not in a Recession’ — Trending Topics

The AI intelligence platform

Every AI story that matters — and the intelligence behind it.

TLDRocket reads all relevant sources, removes duplicate coverage, and publishes a short neutral summary of every story, linking back to the original. Free, no spam, unsubscribe anytime.

Add to Slack

Every story also updates live profiles event timelines weekly rankings the AI Market Index

Friday, 14 February 2025

Fixing Open LLM Leaderboard with Math-Verify

Hugging Face 1 year ago 18

The Open LLM Leaderboard re-evaluated all 3,751 submitted models using Math-Verify, a new evaluation tool that fixes parsing and comparison failures in mathematical reasoning tasks. Models improved by an average of 4.66 points, with algebra-related subsets showing gains of 6.93 to 8.27 points, while some models improved by nearly 90 points on specific subsets. The ranking reshuffling elevated Qwen and DeepSeek models substantially—Qwen scores more than doubled and DeepSeek scores nearly tripled—while Nvidia's AceMath models now dominate the MATH-Hard leaderboard top positions.

Welcome Fireworks.ai on the Hub 🎆

Hugging Face 1 year ago 29

Fireworks.ai is now available as a supported inference provider on Hugging Face Hub, allowing users to run serverless inference on models including DeepSeek-R1, DeepSeek-V3, Mistral-Small, and Llama-3.2-90B. Users can access Fireworks inference through the Hub's website UI, Python and JavaScript SDKs, or HTTP calls, with billing passed through directly at standard Fireworks API rates and no markup from Hugging Face. PRO subscribers receive $2 in monthly inference credits that work across all providers on the platform.

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.