OpenAI and Broadcom unveil LLM-optimized inference chip

OpenAI has spent years as Nvidia's biggest customer, and now it wants to be something else entirely: a chip designer. The company's new partnership with Broadcom has produced Jalapeño, a custom silicon part built from the ground up for one job — running inference on large language models as efficiently as possible.

Inference, not training, is the target here, and that distinction matters. Training a model happens once, in bursts, on massive clusters. Inference happens constantly, billions of times a day, every time someone asks ChatGPT a question. That's where the real operating costs pile up, and it's why a chip tuned specifically for serving already-trained models can move the economics of an AI company more than another training breakthrough would.

Broadcom brings decades of experience building custom silicon for hyperscalers like Google, which has used Broadcom to co-design its TPUs for years. That playbook is now being applied to OpenAI's stack. The pitch from both companies is straightforward: purpose-built hardware can strip out the overhead that comes with general-purpose GPUs, squeezing more tokens per watt and per dollar out of every rack.

This also reads as a hedge against Nvidia's grip on the industry. OpenAI has reportedly signed enormous compute deals with Nvidia, AMD, and now this custom-silicon effort with Broadcom, spreading its bets across suppliers rather than depending on one vendor's roadmap and pricing power. If Jalapeño performs as promised at scale, it gives OpenAI leverage in future negotiations and, eventually, control over its own supply chain — something every major AI lab is quietly racing toward.