Architecture Innovation

32 summarised stories about Architecture Innovation, each linking back to the original source. Browse all topics →

Wednesday, 15 July 2026

Soofi Consortium Releases Soofi S 30B-A3B: An Open Hybrid Mamba-Transformer MoE Foundation Model For German And English

MarkTechPost 19 hours ago

A German research consortium released Soofi S 30B-A3B, an open-source foundation model combining Mamba and Transformer architectures optimized for German and English. The model contains 31.6 billion parameters but activates only 3.2 billion per token, trained on 26.68 trillion tokens with German representing 15.32% of high-quality training data. Soofi S achieved the highest scores among fully open base models tested, scoring 70.1% on English benchmarks and 79.1% on German benchmarks, making it suitable for German document processing, bilingual code assistance, and long-context serving tasks.

Architecture Innovation Foundation Models Multilingual NLP Open Source

Welcome Inkling by Thinking Machines

Hugging Face Blog 1 day ago

Thinking Machines released Inkling, an open-source multimodal language model with 1 trillion parameters that accepts images, text, and audio inputs natively. The model has 975 billion total parameters with 41 billion active at any time, supports a 1 million token context window, and was trained on 45 trillion tokens across multiple modalities. Inkling is available on Hugging Face with immediate support in transformers, SGLang, vLLM, and llama.cpp, enabling developers to build multimodal reasoning applications through fine-tuning or remote inference.

Architecture Innovation Large Language Models Multimodal AI Open Source