Why CPUs still matter in the age of AI agents
The New Stack Frederic Lardinois ● Covered by 2 sources
AI agents are bringing CPUs back into the picture. They’re not just along for the ride anymore; they’re helping run the control flow, sandboxes, and small models.
Based on reporting by The New Stack, Frederic Lardinois — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
When AI infrastructure gets discussed, the spotlight usually swings to GPUs and TPUs. But The New Stack’s conversation with Bhumik Patel of Arm and Mo Farhat of Google makes a different case: as chatbots give way to agents, the CPU is becoming harder to ignore.
Farhat, who leads product management for Axion and Arm-based virtual machines at Google Compute Engine, describes the CPU as an “air traffic controller.” That fits the shift. Chatbots answer. Agents act. They call tools, spin up environments, and execute code, and that creates a mess of branching control flow that CPUs are built to handle.
That doesn’t mean accelerators are out of the picture. Farhat says large language models still typically run on accelerators, but CPUs take on a lot around them: orchestration, data preparation, semantic search, and vector databases. He also says some specialized workloads can run at roughly 25 tokens per second on CPUs, which can be enough for agentic work. Patel adds that Arm is focused on the software and infrastructure layers needed to run these systems at scale, including smaller models such as summarizers, recommenders, and evaluators.
The security piece matters just as much. If agents are writing and running code, they need a place to do it without risking production systems. Patel says that means trusting the generated code, or sandboxing it when trust isn’t there. Google’s answer is gVisor, an open-source isolation layer, and GKE Agent Sandbox, which Farhat says can spin up 300 sandboxes per second per cluster.
Then there’s the cost pitch. Google says Axion can deliver up to 2x the price performance of comparable current-generation virtual machines, with over 60% better energy efficiency. Farhat points to C4A machines for compute-bound jobs where speed matters, and N4A for smaller code-execution tasks where density and cost matter more. His broader point is the simplest one: agentic systems won’t belong to one type of chip. CPU, GPU, and TPU are going to keep sharing the job.
My take — AI-written commentary, not fact-checked reporting
The industry keeps treating CPUs like old furniture just because they’re not flashy. That’s a mistake. Agent systems are turning into orchestration problems with security headaches attached, and those are exactly the sort of chores CPUs still do without acting impressed about it.
Read more about this at: The New Stack