82 summarised stories about Mixture-of-Experts, each linking back to the original source. Browse all topics →
Simon Willison’s Weblog 1 month ago 3 ● 4 sources
Qwen released Qwen3.8-Flash-Next, an open-weights multimodal Mixture-of-Experts model. It is trained on 125B tokens with 6B active parameters. The model is positioned as an early preview of the architecture intended for Qwen4, and users report running it on a DGX Spark with quantized variants up to about 78.9GB.
MarkTechPost 1 month ago 5 ● 2 sources
Z.ai released GLM-5.3-Flash, its first natively multimodal GLM model, featuring a 320B MoE with 18B active parameters per token and a 1,048,576-token context. The model is priced at $0.15 per 1M input tokens and $0.50 per 1M output tokens, and Z.ai says it runs on domestically produced Chinese AI chips during its early “Ox Alpha” testing. This shifts deployment toward API-based usage for most orgs, with self-hosting requiring large hardware (about 306 GiB FP8 weights) while competitors and teams get a lower-cost option for coding and multimodal document/UI workflows.
MarkTechPost 1 month ago 25 ● 4 sources
Alibaba’s Qwen team released Qwen3.8-Flash-Next, an open-weight multimodal Mixture-of-Experts model aimed at low cost per token. It uses a 125B backbone plus 51B N-gram embeddings and a 4B multi-token prediction module, with only 6B parameters active per token. The update introduces a Gated DeltaNet/Qwen Sparse Attention hybrid attention stack plus gated residuals, N-gram embeddings, and the Muon optimizer, lowering reported training cost to about one-ninth of Qwen3.7-Plus.
The daily briefing
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.
TLDRocket
15 October 2026
Meta used today’s open-weight model drop to make a point: “frontier” doesn’t have to stay in the datacenter. Muse Spark Open ships as an 816B multimodal model with a 1 million–token context window, paired with Muse Code for coding and agent-style workflows—and it comes with strings. The license is free for use and fine-tuning, but it adds revenue-linked conditions (including advance notice above $20 million) and asks developers to display “Built with Muse Spark Open.” Running locally is a hardware project in itself: Meta says you need about 1TB of RAM. Still, the rollout is broad—Meta is making it available via multiple API platforms for desktop-focused tasks, and now through Amazon SageMaker JumpStart for deploy-to-your-own-AWS-account use, including BF16 and a 4-bit NVFP4 variant.
Read the full briefing →