Intel squeezed a 1.58-bit LLM down to 1.485 bits without changing a single weight

Intel researchers have taken a ternary language model below the neat little 1.58-bit floor that usually gets quoted for three-valued weights. Their trick did not touch the model itself. Instead, it changed how the weights are stored, using a format called BITCOS to squeeze one checkpoint down to 1.485 bits per weight while also improving decoding throughput by up to 18% on CPUs and 27% on GPUs.

The headline number has always been cleaner than reality. Ternary models use -1, 0 and +1, but real checkpoints are not evenly split across those values. Intel looked at 29 checkpoints from seven ternary model families and found zeros ranging from 29.7% to 51.5%. In 26 of those checkpoints, there were enough zeros for BITCOS to beat the usual five-trit packing scheme.

BITCOS stands for “BITmap and COmpacted Signs,” and the name gives away the method. One bit marks whether each weight is zero or not, and only the nonzero weights get a sign bit. That means a zero takes less space than a positive or negative weight. Because the format only changes storage, unpacking restores the original -1, 0 and +1 values without changing accuracy. Intel says BITCOS beats five-trit packing once more than 37.5% of weights are zero.

The sparsest case in the paper was a ternary version of Qwen3-1.7B made with CAT-Q post-training quantization. It had 51.48% zero weights, which let BITCOS reach 1.485 bits per weight. Intel also built unpacking kernels for AVX-512 and AVX2 CPUs and for Xe2 GPUs, aiming at token-by-token decoding with small batch sizes where memory traffic matters a lot.

The speedups were real, but not universal. On a 64-core Xeon server, BITCOS ran 10% to 18% faster than the 2-bit kernels. On a 24-core Core Ultra 9, gains were 2% to 15%. The integrated Arc 140V saw 9% to 22% improvement, and the discrete Arc Pro B70 saw 2% to 27%. But on the eight-core Lunar Lake CPU, Intel’s fixed 2-bit kernel won every time because bandwidth was plentiful and unpacking became the bottleneck.

The paper is still unreviewed, and the tests stayed inside Intel’s own hardware. That keeps the result interesting rather than settled. It is another reminder that AI efficiency gains often come from boring storage tricks, not magical new models. Boring is good. Boring ships.