The Sequence AI of the Week #899: Inside Inkling: A Trillion-Parameter Model That Only Wakes Up 41 Billion at a Time

Inkling is the kind of model that makes raw parameter counts almost meaningless as a headline stat. Yes, it's built from 975 billion parameters, close enough to a trillion that only pedants would argue the difference. But the number that actually explains how the thing behaves is 41 billion — the slice of the model that activates for any single token passing through it. That's about 4.2 percent of the total capacity doing real work at any given instant.

The architecture behind this is a mixture-of-experts setup, and the university analogy fits better than most technical descriptions. Picture a campus with 256 specialist departments per relevant layer. A token walks in, and instead of convening the whole faculty, a router picks six departments that seem relevant plus two generalists who always show up, blends their output, and sends the token on its way. A snippet of Python code might trigger a different set of departments than a phrase in Greek or a caption on an image. The routing decision happens fresh for every token, which is why the same model can feel like dozens of narrower models stitched together on demand.

What this buys Inkling is a strange kind of efficiency. The compute cost per token stays low because you're only running a sliver of the network's parameters through the math. But the storage, the memory bandwidth, and the networking requirements don't shrink to match — you still have to keep the entire trillion-ish-parameter warehouse loaded and addressable, because you never know in advance which departments a given token will need. So the sparse arithmetic hides a very unsparse infrastructure bill.

This is the tradeoff that mixture-of-experts models have been chasing for a few years now, and Inkling is a fairly extreme point on that curve. Dense models spend every parameter on every token, which is simple but expensive. Sparse models like this one try to get the quality benefits of massive scale while keeping the per-token compute closer to what a much smaller model would cost. The catch is that building and serving something like this requires infrastructure that can handle the full trillion-parameter footprint even though most of it sits idle at any moment — which is a very different engineering problem than just training a bigger dense network.