The CPU Comeback Is Upon Us
IEEE Spectrum Matthew S. Smith ● Covered by 2 sources
AWS is telling engineers to save CPU cycles as AI agents eat server capacity. The surprise: the AI boom may be pushing CPUs back to center stage, not just GPUs.
Based on reporting by IEEE Spectrum, Matthew S. Smith — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Amazon Web Services has reportedly told engineers to conserve CPU cycles at all costs. That’s a sharp turn for a cloud business that spent the early AI boom worrying far more about GPUs, then memory, than about plain old CPUs. Now the wait for CPU server capacity is getting long enough that AWS seems to have been caught off guard.
The reason is agentic AI. These systems don’t just generate text; they call tools, hit APIs, open files, move code around, and then call more tools. Matt Kimball of Moor Insights & Strategy says the scaling problem is brutal: an enterprise rollout that starts with 100 agents can turn into tens of thousands, hundreds of thousands, or even millions once those agents begin spawning sub-agents and talking through Anthropic’s model context protocol.
That shifts a lot of the work onto CPUs. Intel’s Souvik Kundu says the CPU handles parsing, choosing tools, making API calls, running code, collecting results, and sending them back. AMD’s Madhu Rangarajan says that in testing, seven of eight stages in realistic agentic AI pipelines ran entirely on the CPU. Kundu’s work with Georgia Tech researchers suggests scheduling tweaks can cut end-to-end latency by up to 1.8x under sustained load, but agentic systems keep multiplying the work faster than simple fixes can absorb it.
There’s another catch: tokenization. Euijun Chung, a PhD student at Georgia Tech, and his co-authors found that when a server has too few CPU cores, it falls behind dispatching work to GPUs, which then sit idle waiting for instructions. As sequences get longer, tokenization gets more painful, especially when an agent keeps adding tool results back into the prompt. In longer test runs, more CPU cores cut time-to-first-token latency by roughly 1.5x to 7x.
That helps explain why the hardware market is shifting. Intel has sold out of server CPUs through at least the end of the year, AMD has doubled its server CPU forecast, Arm and Qualcomm have launched new CPUs aimed at agentic AI, and Nvidia has put its own Arm-based Vera CPU at the center of the Vera Rubin platform. Kimball’s read is blunt: CPUs are no longer the afterthought in AI infrastructure.
My take — AI-written commentary, not fact-checked reporting
The industry spent two years talking as if AI meant “buy more GPUs” and call it strategy. Now the bill for all the boring parts — parsing, tokenizing, checking, dispatching — is coming due, and it wants CPUs. That’s the kind of correction that usually arrives with higher prices and a few embarrassed PowerPoint decks.
Read more about this at: IEEE Spectrum