Timing Trick Cuts Energy Used in LLM Training by Up to 14 Percent
IEEE Spectrum Dina Genkina
Dutch researchers found a way to cut LLM training energy use by up to 14% without slowing things down. They just fine-tune the GPU's internal clock speeds, kernel by kernel.
Based on reporting by IEEE Spectrum, Dina Genkina — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Training frontier AI models eats an absurd amount of electricity. GPT-4 reportedly took around 50 gigawatt-hours to train back in 2023, roughly what 5,000 American homes burn through in a year. Every model since has almost certainly needed more. So when a small research group finds a way to shave that number down without touching accuracy or speed, it's worth paying attention.
A team at the University of Twente, led by PhD candidate Jeffrey Spaan, has done exactly that. Presenting at the Computing Frontiers conference in Catania last month, they showed that carefully adjusting a GPU's clock frequency during training can cut energy use by up to 14 percent while slowing computation by less than one percent. The trick isn't new in concept. It's called dynamic voltage and frequency scaling, or DVFS, and chip designers have used versions of it since the 1990s. What's new is how finely Spaan's team applied it.
GPUs run on two clocks: one for the computing core, one for memory. When the core is crunching numbers, it needs full speed, but the memory clock can idle down since it's not doing much. Flip that around when the core is waiting on data from memory, and the memory clock can ramp up while the core slows to a crawl. Earlier attempts to exploit this only adjusted frequencies once per training iteration, splitting the forward pass from backpropagation. Spaan went granular instead, tuning frequencies at the level of individual kernels, the small computational chunks a GPU processes in parallel. A single neural network layer, in his setup, breaks into roughly 40 of these kernels, and adjusting each one separately is where the real savings showed up.
The team tested this on GPT-3-XL, a 1.3 billion parameter model, running on an Nvidia RTX 3080 Ti, and focused on a single layer to keep the experiment manageable. That's a modest scale compared to the systems training today's largest models, and the 14 percent figure is a best-case number since the researchers didn't account for the time it takes to actually switch clock frequencies mid-run. Spaan says newer chips like Nvidia's Blackwell GPUs switch fast enough that they should capture most of the theoretical savings, but older hardware might not.
What makes this interesting isn't the specific percentage. It's the acknowledgment that GPUs already try to do this automatically, and fail, because the chip can't predict which kernel is coming next. Spaan's method works because it has foresight the hardware itself lacks. His team is now building a tool to automate the frequency tuning for arbitrary workloads, betting that if the energy savings come free of any performance cost, industry will actually bother adopting it.
My take — AI-written commentary, not fact-checked reporting
This is the unglamorous kind of AI research that actually deserves attention, because nobody is going to slow down model scaling for climate reasons, but a free 14 percent efficiency gain with zero performance cost is the sort of thing hyperscalers will grab instantly once it's packaged into a tool. I'd rather see a dozen boring hardware-level wins like this than another splashy benchmark chart, since the real emissions problem was never going to be solved by anyone voluntarily training smaller models.
Read more about this at: IEEE Spectrum
