The Open Arabic LLM Leaderboard 2

Hugging Face Blog · 1 year ago

The Open Arabic LLM Leaderboard released a second version in December 2024, replacing machine-translated and saturated benchmarks with natively developed Arabic tasks and newly available datasets. The updated leaderboard added Native Arabic MMLU (40 tasks with 15,000 questions), Human Translated MMLU (57 tasks), MedinaQA, AraTrust (522 questions on safety), and ALRAGE, while retaining 6 AlGhafa tasks and 2 Belebele tasks from the first version. The revised benchmarks aim to better capture Arabic's morphological complexity and cultural context, addressing community feedback that prior datasets failed to reflect real-world Arabic use cases and dialect diversity.