Alibaba’s Qwen Team Releases Qwen3.8-Flash-Next: A 125B Multimodal MoE With 6B Active Parameters Previewing the Qwen4 Architecture
MarkTechPost Asif Razzaq ● Covered by 4 sources
Alibaba’s Qwen team dropped Qwen3.8-Flash-Next, a big open-weight multimodal model with only 6B active parameters. It’s a preview of Qwen4, but you’ll need serious hardware to run it.
Based on reporting by MarkTechPost, Asif Razzaq — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Alibaba’s Qwen team has put out Qwen3.8-Flash-Next, an open-weight multimodal MoE model built with one blunt goal: lower cost per token. Under the hood, it combines a 125B backbone, a 51B N-gram embedding table, and a 4B multi-token prediction module. On disk, that adds up to 180B parameters, but only 6B wake up for any given token.
The release is being framed as an architectural preview of Qwen4, much like Qwen3-Next served as a preview for Qwen3.5. Four pieces define the new setup: a hybrid of Gated DeltaNet and Qwen Sparse Attention, a Gated Residual design, N-gram Embedding, and the Muon optimizer. The model’s layer pattern is 12 repeats of 3 GDN layers followed by 1 QSA layer, across 48 layers total, with sparse attention working at micro-block granularity instead of token-by-token.
Qwen says the training cost came in at about one-ninth of Qwen3.7-Plus. That’s the headline number, and it’s the one that matters if the company can keep repeating it. Serving the thing is another story. The FP8 checkpoint is 172.78 GiB, the BF16 version is 335.28 GiB, and the minimum validated FP8 setup in vLLM is TP2 on GB300, with TP4 recommended. On an 8×H200 node, Qwen says TEP8 is the right path, because plain TP8 clashes with the checkpoint’s 128-wide quantization blocks.
The model is also very much a real product and not just a paper architecture. It already powers Standard mode on QwenWork, works with Qwen Code, and can be served through vLLM, SGLang, TokenSpeed, transformers serve, and llama.cpp for GGUF quants. Fine-tuning support is listed for Unsloth, Swift, and LLaMA-Factory. Thinking mode is on by default, with reasoning_effort set to xhigh, medium, or low.
Benchmarks show a model that is strong in several practical areas, especially code and multimodal work, but not the best everywhere. Qwen reports 91.9 on LiveCodeBench v6, 88.5 on RealWorldQA, and 95.7 on MathVision with a code interpreter. It still trails Claude Opus 4.6 on HLE and DeepSeek-V4-Flash-0731 on NL2Repo-Bench. That leaves the familiar gap in place: efficiency is moving fast, frontier reasoning less so.
My take — AI-written commentary, not fact-checked reporting
This is the part of the AI race that actually matters: not who can post the loudest benchmark, but who can ship a model that costs less to run without collapsing into toy territory. Qwen is pushing hard on that front, and the multi-GPU reality check is refreshingly honest. The hype machine loves “open-weight”; the bill for 335.28 GiB of BF16 does not.
Read more about this at: MarkTechPost