Why Performance per Watt Is the Ultimate Metric for AI Infrastructure Efficiency
NVIDIA Shruti Koparkar
Nvidia says power, not chip count, is now the real limit on AI factories, so performance-per-watt is the metric that matters most. Its GB300 NVL72 claims up to 25x better efficiency than the old Hopper generation on some models.
Based on reporting by NVIDIA, Shruti Koparkar — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Every AI company likes to talk about raw compute, but Nvidia's latest pitch is about something less flashy and far harder to fake: how much useful work you squeeze out of every watt. The argument goes like this — since power availability caps how many tokens an AI factory can crank out, and tokens are what generate revenue, the company that gets the most output per watt wins, full stop. It's a metric that can't be marketed around, only earned by actually running workloads at scale.
The technical crux is domain size — how many GPUs are wired together over a fast interconnect to act as one big serving unit. Nvidia's older Hopper chips topped out at eight-GPU domains. The Blackwell NVL72 platform pushes that to 72, which matters a lot for today's mixture-of-experts models that dominate the frontier AI landscape. Nvidia claims GB300 NVL72 delivers up to 25x the performance per watt of Hopper on DeepSeek V4 Pro, 20x on GLM5.1, and 10x on Kimi K2.6, a model built for long-running agentic tasks. Those numbers come from SemiAnalysis InferenceX benchmarking, and Nvidia is careful to frame them as a floor, not a ceiling — a starting point Blackwell keeps improving on.
What's notable is how much of that improvement comes from software rather than new silicon. Nvidia says performance per watt on DeepSeek V4 jumped up to 5x in a single month, driven by tricks like NVFP4 quantization, disaggregated serving, and smarter routing across the inference stack — tools like Dynamo, TensorRT LLM, SGLang and vLLM. There's also a less glamorous but very real problem: cooling and rack-level losses mean only about 60% of grid power actually turns into useful compute. Nvidia's DSX MaxLPS software attacks that by shuffling power dynamically between GPUs and racks, reportedly letting operators run up to 40% more GPUs within the same power envelope.
But the real signal isn't the benchmark chart, it's who's actually running this stuff in production. Anthropic, OpenAI and SpaceXAI are named as Blackwell NVL72 users for inference. CoreWeave has Kimi K2.6 running on GB300 NVL72 with speculative decoding tricks layered in. Perplexity serves millions of daily queries off Qwen3 variants on GB200 NVL72, and Fireworks AI runs GLM 5.2 for customers including Cursor and Factory AI. Production reliability at rack scale is genuinely different from single-node deployments — new failure modes show up that you only learn to handle by living through them — and Nvidia is leaning hard on that accumulated experience as the thing that gives its next platform, Vera Rubin, a head start before it even ships.
My take — AI-written commentary, not fact-checked reporting
Nvidia turning power efficiency into the headline metric is a smart bit of narrative control — of course the company selling the racks wants everyone benchmarking rack-scale systems instead of raw chip specs. That said, the production list here is the part worth watching, not the Pareto curves: when Anthropic, OpenAI, CoreWeave and Perplexity are all quietly standardizing on the same 72-GPU domain architecture, that's less marketing and more a sign the industry has already decided the question for itself. The bigger worry nobody's pricing in yet is that if AI's growth ceiling really is grid power rather than chip supply, the fight over energy access is about to get a lot uglier than the fight over GPUs ever was.'
Read more about this at: NVIDIA