Soofi Consortium Releases Soofi S 30B-A3B: An Open Hybrid Mamba-Transformer MoE Foundation Model For German And English

MarkTechPost · 3 months ago · 34

A German research consortium released Soofi S 30B-A3B, an open-source foundation model combining Mamba and Transformer architectures optimized for German and English. The model contains 31.6 billion parameters but activates only 3.2 billion per token, trained on 26.68 trillion tokens with German representing 15.32% of high-quality training data. Soofi S achieved the highest scores among fully open base models tested, scoring 70.1% on English benchmarks and 79.1% on German benchmarks, making it suitable for German document processing, bilingual code assistance, and long-context serving tasks.