A guide to setting up your own Hugging Face leaderboard: an end-to-end example with Vectara's hallucination leaderboard

Hugging Face · 2 years ago · 45

Vectara released a new leaderboard using Hugging Face's open-source template to track how frequently large language models generate hallucinations in document summaries. The leaderboard evaluates models including GPT-4, Gemini, Llama 2, and Mistral 7B using Vectara's Hughes Hallucination Evaluation Model (HHEM), which was named in honor of a deceased colleague. The template-based approach enables community members to submit new models for evaluation and allows the leaderboard to update dynamically as results arrive.