Featuring Every Eval Ever Results on Hugging Face Model Pages

Anyone who has tried to compare AI benchmark scores across papers, leaderboards, and blog posts knows the numbers rarely line up. LLaMA 65B has been reported at both 63.7 and 48.8 on MMLU, depending on who ran the eval and how. That gap isn't a rounding error, it's a symptom of how scattered and underspecified AI evaluation reporting has been. Hugging Face and the EvalEval Coalition are now tackling that mess from two directions at once, and as of this week their systems talk to each other.

Every Eval Ever, or EEE, launched in February 2026 as the Coalition's answer to standardizing how evaluation results get written down. It's a single JSON schema that captures who ran a benchmark, which model version, how it was accessed, the generation settings used, and what the metric is actually measuring, with an optional per-sample JSONL file for anyone who wants the raw outputs. Hugging Face's own Community Evals, which launched the same month, took a different approach: decentralize benchmark reporting directly on the Hub, so scores live in YAML files inside model repos and feed automatically into leaderboards tied to specific datasets.

The new piece is a converter that moves data between the two without anyone hand-copying results twice. Feed it an EEE datastore collection and it maps fields like hf_repo, evaluation_name, and score straight into the small YAML format Hugging Face expects, then checks what's already on a model's main branch and open pull requests before touching anything. Scores get flagged as already present, in conflict, or ready to go, and nothing gets pushed until you type OPEN PRS yourself. Right now it only handles four benchmarks — MMLU-Pro, GPQA, HLE, and GSM8K — but that list is expected to grow.

The payoff is a two-way link: a score that shows up on a Hugging Face model card now carries a badge pointing back to the full EEE record, with the harness version, generation config, and reproducibility notes intact. Submit through an official organization account and the result gets a verified checkmark on EvalEval, telling readers the number came straight from the source rather than a third party's rerun. The datastore behind all this has already grown to roughly 229,000 results across more than 22,000 models and 2,200 benchmarks, pulled from 31 different reporting formats — reproducing that from scratch would run into the hundreds of thousands of dollars, which is as good an argument as any for not letting it scatter again.

What's notable here isn't the schema itself, which is fairly unglamorous plumbing, but the fact that two separate efforts with overlapping goals decided to interoperate instead of competing for the same niche. Eval reporting has been a mess for years precisely because everyone builds their own format and leaderboard. This is a rare case of infrastructure consolidating before it fully fragmented.