The Sequence Knowledge- Issue 924: The Distilled Models You Need to Know About

PrismML’s Bonsai 27B is a neat little insult to the old idea that big parameter counts must mean big hardware. A 16-bit 27-billion-parameter model needs about 54 gigabytes just to hold its weights. Bonsai comes in a ternary version at roughly 5.9 gigabytes and a binary version at roughly 3.9 gigabytes, with the smaller one built to fit the memory budget of a high-end phone.

That alone would be enough to make it interesting, but the more useful detail is what Bonsai keeps. It is multimodal. It supports long context. And PrismML says it preserves much of the reasoning and tool-use behavior of the full-precision Qwen3.6-27B model it came from. So this is not just about packing fewer bits into fewer bytes. It is about dragging useful behavior across a very tight size limit.

Bonsai is also a reminder that the clean labels people use for model compression are getting harder to defend. PrismML’s public story leans on end-to-end low-bit training and quantization, not a classic teacher-student setup. That makes Bonsai less like a textbook distillation example and more like a border case where distillation, pruning, quantization-aware training, and systems work all blur together.

And that may be the real point. The important object in AI is starting to look less like a single checkpoint and more like a lineage: one model discovers a capability, another learns its probability landscape, another inherits reasoning traces, a pruned descendant inherits the shape, and a low-bit version turns the whole thing into something deployable. The family tree matters more than the family photo.