PrismML hopes its tiny LLM will change how we all use AI

PrismML is trying to make a basic AI assumption look outdated: that smarter models have to be huge. The startup just released Bonsai 2 27B, a compressed version of Alibaba’s Qwen3.8 27B that now fits into 5.9 GB of memory. That is a major squeeze — roughly 9x to 10x smaller than the original — and it’s small enough to run on a PC, with a high-end phone potentially in reach too.

The company isn’t a household name yet, but the people behind it are unusually well connected. PrismML was founded by Caltech researchers and is led by Babak Hassibi, a Caltech professor who specializes in compression. Ion Stoica is advising the company, and PrismML also has backing from Khosla Ventures, Cerberus Capital, and Caltech. There’s also talk, which Hassibi wouldn’t confirm, that Apple may be interested.

What makes PrismML stand out is not just the size of its models, but how close they stay to the originals. Hassibi says Bonsai 2 reaches 98% of Qwen’s aggregate benchmark scores. The earlier Bonsai model, released in March, reached 95%. That first version has already been downloaded more than 11 million times, and PrismML says its even smaller models have picked up another 2.6 million downloads.

The trick is in the weights. Instead of storing each one with 16 bits, PrismML uses what it calls ternary weights: +1, −1, or 0. That cuts the memory load hard. And the company thinks it can go further, with plans to compress models in the several-hundred-billion-parameter range over the next couple of months.

For Hassibi and Stoica, the point is clear: if the model runs on your own device, it becomes both private and free in the everyday sense — no cloud bill, no data shipping off to someone else’s servers. That’s a much more interesting pitch than another bigger model nobody can actually use without renting a datacenter.