Generative Models

27 summarised stories about Generative Models, each linking back to the original source. Browse all topics →

Wednesday, 21 May 2025

Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance

Hugging Face Blog 1 year ago

Abu Dhabi's Technology Innovation Institute released Falcon-H1, a series of six open-source language models combining Transformer attention with State Space Model architecture to improve efficiency and long-context performance. The smallest model, Falcon-H1-0.5B, delivers performance comparable to typical 7B models from 2024, while the 34B variant matches or exceeds much larger competitors like Qwen2.5-72B on long-context tasks. The hybrid design enables faster inference and lower memory usage across deployments ranging from edge devices to large-scale systems, with support for 18 languages and 256K context length.

Architecture Innovation Efficiency & Optimization Generative Models Open Source

Falcon-Arabic: A Breakthrough in Arabic Language Models

Hugging Face Blog 1 year ago

The Technology Innovation Institute released Falcon-Arabic, a 7-billion-parameter language model adapted from Falcon 3 to support Arabic, English, and other languages. The model achieved top performance on the OALL v2 benchmark suite, outperforming Arabic language models of similar size and surpassing models up to four times larger on tasks like Arabic MMLU and MadinahQA. Falcon-Arabic enables Arabic-language applications in chatbots, content generation, and document understanding while supporting both Modern Standard Arabic and regional dialects.

Algorithm Innovation Arabic NLP Benchmarking Generative Models