Timing Trick Cuts Energy Used in LLM Training by Up to 14 Percent
IEEE Spectrum AI Dina Genkina
Researchers at University of Twente demonstrated that adjusting GPU clock frequencies during different computational stages of language model training can reduce energy consumption by up to 14 percent. The technique, called dynamic voltage and frequency scaling, was applied at a finer granularity than previous attempts, adjusting frequencies per kernel rather than per training iteration, with the experiment training GPT-3-XL showing 14 percent energy savings while increasing training time by only 0.6 percent. The team is now developing a tool to implement optimal frequency scaling automatically, with adoption depending on whether the energy savings justify the modest performance trade-off for industry users.
Why it matters
OpenAI’s fourth large language model (LLM), GPT-4, took an estimated 50 gigawatt-hours to train, or the equivalent of 5,000 American homes’ yearly power consumption. That was in 2023. Since then, the computational resources used to train frontier LLMs have only increased, though direct power usage numbers are hard to come by.Now, a research group at the University of Twente in the Netherlands has shown that you can save up to 14 percent of the energy used in LLM training without sacrificing speed by cleverly adjusting the clock frequency of the GPU during computation. Jeffrey Spaan, Ph.D. candidate at University of Twente and lead author on the article, presented the results at the Computing Frontiers conference in Catania, Sicily, last month.“My research is about finding computing waste,” Spaan says. “It’s similar to underutilization of the hardware, but instead of optimizing the software for the hardware, we try to optimize the hardware for the software.”Making the GPU tick Spaan and
