How controllers from industrial machinery can coordinate multitask machine learning

Training a model to hit several objectives at once has always been a headache, and the standard fix — blending every objective's gradient into one averaged update at each step — turns out to be part of the problem. Researchers from Amazon Web Services and Carnegie Mellon, presenting at this year's ICML, argue that forcing objectives to compromise on every single step causes three distinct failures: conflicting gradients can actively degrade each other (disagreement), objectives that mattered early on become dead weight later but weights don't adapt fast enough (drift), and adaptive schemes sometimes starve an objective down to near-zero weight so nobody can tell if it ever did anything (drought).

Their answer, called ControlG, comes from an unlikely place: industrial control theory. Instead of mixing gradients every step, the system dedicates full computational capacity to one objective at a time, and a proportional-integral-derivative controller — the same kind of feedback loop that regulates cruise control or espresso machine water temperature — decides which objective gets attention next. The team found that even naive random scheduling, picking objectives at random for blocks of training steps, often beat sophisticated gradient-manipulation baselines like AutoSSL and ParetoGNN on node clustering tasks, simply because separating objectives in time kills instantaneous gradient conflict.

ControlG builds on that insight with three loops running at different speeds. A slow 'sense' loop estimates how hard each objective currently is, using a spectral-demand measure of how much neighboring nodes disagree on updates, plus an interference score borrowed from multiple-gradient descent that flags which objectives are constraining trade-offs. An epoch-level 'plan' loop turns those difficulty readings into a target allocation of compute across objectives, leaning on a log-hypervolume metric to prioritize whichever objective is lagging furthest from the Pareto front. Then a block-level PID controller tracks that plan in real time, correcting for objectives falling behind schedule and damping oscillations along the way.

On nine graph benchmarks covering homophilic networks like Cora and PubMed, heterophilic ones like Chameleon and Squirrel, and the 169,000-node ogbn-arxiv, ControlG posted average ranks of 1.4, 1.9, and 1.8 across node classification, link prediction, and clustering — ahead of every baseline tested. The gains were concrete: 1.5 percentage points over CAGrad on Cora, 1.1 over PCGrad on PubMed, 1.8 over CAGrad on Coauthor-CS, and a 1.2-point jump to 72.86% accuracy on ogbn-arxiv versus CAGrad's 71.62%. It wasn't free — ControlG adds 16 to 31 milliseconds per step compared to 8 to 15 for plain scheduling — but that's still far cheaper than AutoSSL's 125 to 414 milliseconds or ParetoGNN's up to 764.

Ablation tests showed every piece of the system earns its keep: stripping out the planner and just allocating uniformly cost up to 3.4% on some datasets, removing the difficulty signals hurt similarly, and swapping the PID controller for simple random sampling from the plan dropped performance by 1 to 2%. Beyond the numbers, the researchers point out that ControlG leaves behind a readable training log — scheduling timelines, deficit traces, difficulty trajectories — that can show which objective actually shaped a model when something downstream goes wrong. They're now looking at whether the same sense-plan-control loop could untangle the competing instruction-following, reasoning, and safety objectives that fight for space during LLM fine-tuning.