Gimlet Labs nabs $300M for its disaggregated inference platform
SiliconANGLE Maria Deutscher ● Covered by 2 sources
Gimlet Labs raised $300M at a $3B valuation for software that speeds up AI inference. It splits model work across the right chips, and that’s now pulling in huge orders.
Based on reporting by SiliconANGLE, Maria Deutscher — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Gimlet Labs has closed a $300 million Series B at a $3 billion valuation, bringing its total outside funding to $392 million. Andreessen Horowitz led the round, with Arm Holdings, Samsung Ventures, Microsoft’s M12 and more than a dozen other backers joining in.
The startup is betting that inference gets faster when the model’s pieces stop pretending they all need the same hardware. A large language model is made up of multiple software modules, and those modules stress chips in different ways. Some lean hard on memory. Others don’t. Gimlet’s platform breaks an LLM into its module-level parts and puts each one on the chip architecture that fits best.
That idea builds on disaggregation, a trend that already splits work across chips. The common version divides prefill and decode, the two main phases of inference. Gimlet goes further than that and supports more granular setups, including cases where decode itself is split into smaller workflows. It also covers approaches where a lightweight drafter model makes an early pass and a frontier model cleans it up, with each model running on a different chip architecture.
The company says its software cuts down the effort needed to set up those workflows and tunes each model module for the hardware it lands on. It does that with AI agents and a custom compiler. The agents explore different ways to adapt code, test the best-looking option, and the compiler applies both general and chip-specific optimizations.
Gimlet sells the software as a serverless product and as a managed service for enterprises running their own infrastructure. It says it has already taken in billions of dollars in customer orders, and in March said its customer list included one of the world’s largest cloud providers and a top three AI lab. The new cash will go toward expanding the serverless capacity by several hundred megawatts and pushing into custom hardware, including an inference-optimized server that doesn’t use a motherboard and can run outside data centers.
My take — AI-written commentary, not fact-checked reporting
This is what real AI infrastructure looks like: less chatbot theatre, more chip plumbing. The market keeps rewarding whoever can shave waste off inference, which is far less glamorous than model demos and a lot harder to fake. Bigger hardware bets are back, because the bill always shows up after the hype does.
Read more about this at: SiliconANGLE