Introducing the Open Leaderboard for Hebrew LLMs!

Hugging Face · 2 years ago · 43

A new open leaderboard has been created to evaluate large language models specifically for Hebrew, addressing gaps in existing benchmarks that fail to account for the language's morphological complexity. The leaderboard includes four evaluation tasks: Hebrew question answering, sentiment analysis, pronoun resolution (Winograd Schema Challenge), and English-Hebrew translation, with models automatically deployed and assessed via HuggingFace's Inference Endpoints. This creates a platform for researchers to submit and compare Hebrew language models while identifying areas for improvement in Hebrew NLP development.