Nvidia ties AI factory economics to tokens and power efficiency
SiliconANGLE Victoria Gayton
Nvidia says AI factories are now judged by tokens and power use, not just chips. That makes the whole data center, and every watt, part of the product.
Based on reporting by SiliconANGLE, Victoria Gayton — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Nvidia is trying to reframe AI infrastructure as something closer to an industrial plant than a pile of servers. Ian Buck, the company’s vice president and general manager of hyperscale and HPC, said the real unit of value is now the token: the output of inference, the place where deployed models turn requests into revenue.
That sounds abstract until you listen to the economics underneath it. Buck’s point was that AI systems don’t just sit still after training. Companies keep updating them, aligning them, and feeding them more data, so the line between training and inference keeps blurring. The work doesn’t stop once a model ships. It keeps getting tuned in production.
And then there’s latency. For some workloads, the fastest possible reasoning is worth paying for, which creates a second tier of inference demand. Buck said Nvidia’s Groq 3 LPX inference accelerator can be boosted on top of its Vera Rubin platform to raise per-user token rates for time-sensitive jobs, with fintech among the areas showing interest because things happen in real time.
The other hard limit is power. Buck said data centers run into a natural cap there, and that cap forces the industry to care about tokens per watt. Nvidia says each GPU generation is designed to deliver more than 10 times the efficiency, and Buck pointed to Blackwell as a case where the company saw a 30x improvement in tokens per watt.
That pushes the conversation beyond chips alone. Buck also pointed to CoreWeave as an example of a partner that can help customers choose configurations or higher-level inference services without getting buried in options. The message is simple: AI infrastructure is becoming a system business, and the best systems are the ones that turn power into useful tokens with the least waste.
My take — AI-written commentary, not fact-checked reporting
This is the part of AI people keep trying to skip: electricity always wins. The industry loves talking about clever models, but the bill arrives in watts, not vibes. Nvidia is smart to turn that into the headline metric before someone else does.
Read more about this at: SiliconANGLE