Thinking Machines Lab Releases Inkling-Small: A 276B Total, 12B Active Open Weights Multimodal MoE Model
MarkTechPost Asif Razzaq ● Covered by 33 sources
Thinking Machines Lab dropped Inkling-Small, a 276B MoE model with only 12B active params, and it beats its own 975B parent on several benchmarks. The quantized version fits on one GPU, which is the actual news here.
Based on reporting by MarkTechPost, Asif Razzaq — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Thinking Machines Lab just did something a little unusual: it released a smaller model that outperforms its own flagship on a chunk of the benchmarks that matter. Inkling-Small carries 276B total parameters against Inkling's 975B, but only 12B are active per token thanks to a sparse Mixture-of-Experts design routing to 6 of 256 experts plus 2 shared ones. On SWE-bench Verified it hits 80.2% versus Inkling's 77.6%. On Terminal-Bench 2.1 it reaches 64.7%. ARC-AGI-2 climbs to 40.1% from 36.5%. For a model roughly a quarter the size, that's not a rounding error.
The real headline, though, is deployability. The BF16 checkpoint still wants 600GB of aggregated VRAM, which means four B300s or eight H200s — fine for a lab, painful for anyone else. But the NVFP4 quantized version drops that floor to 180GB, low enough to run W4A4 on a single NVIDIA B300 or W4A16 across two H200s. That's the difference between a model living exclusively inside frontier labs and one a well-funded startup can rent for an afternoon. Apache 2.0 licensing on Hugging Face, support across SGLang, vLLM and Unsloth, and compatibility with existing H200 fleets means banks, hospitals and telecoms now have a private-weights option that doesn't require buying new silicon.
Architecturally it's a 42-layer decoder that's encoder-free and natively multimodal — no separate vision tower bolted on afterward. Images get chopped into 40x40 patches and run through a four-layer hMLP; audio comes in as dMel spectrograms sampled at 16kHz. Both get embedded alongside text tokens and processed in the same stack, up to a 1M token context window. Training happened on NVIDIA's GB300 NVL72 systems, and because Inkling-Small started training after its big sibling, the team could revise the data mix and post-train using on-policy distillation with Inkling itself as teacher, followed by two more weeks of agentic coding reinforcement learning.
Not everything went up. SimpleQA Verified, a pure factual-recall benchmark, fell to 20.6% from Inkling's 43.9%, and the AA Omniscience index went negative. Tau 3 Banking dropped too. Thinking Machines is candid about this in the model card, framing it as a tradeoff of the smaller parameter count rather than a training flaw, and it explicitly says the model adds no meaningful uplift risk beyond what's already available in open weights, recommending third-party moderation like Llama Guard for anything consumer-facing. That kind of plain-spoken safety framing, paired with hard numbers instead of vague reassurance, is rarer than it should be in this space.
My take — AI-written commentary, not fact-checked reporting
What's actually newsworthy isn't the benchmark leapfrogging — smaller distilled models beating their teachers on select evals happens all the time — it's that a 276B-parameter model with real multimodal chops now fits on a single GPU you can rent by the hour. That's the line between labs flexing and infrastructure people can actually build on, and Thinking Machines just moved it. I'll also give them credit for publishing the SimpleQA regression instead of burying it; that kind of honesty about tradeoffs is exactly what open-weight releases need more of, and exactly what a lot of closed-model marketing conveniently skips.”
Read more about this at: MarkTechPost