Timing Trick Cuts Energy Used in LLM Training by Up to 14 Percent

IEEE Spectrum AI · 1 month ago

Researchers at University of Twente demonstrated that adjusting GPU clock frequencies during different computational stages of language model training can reduce energy consumption by up to 14 percent. The technique, called dynamic voltage and frequency scaling, was applied at a finer granularity than previous attempts, adjusting frequencies per kernel rather than per training iteration, with the experiment training GPT-3-XL showing 14 percent energy savings while increasing training time by only 0.6 percent. The team is now developing a tool to implement optimal frequency scaling automatically, with adoption depending on whether the energy savings justify the modest performance trade-off for industry users.