How controllers from industrial machinery can coordinate multitask machine learning
Amazon Science
Amazon researchers developed ControlG, a framework that uses proportional-integral-derivative (PID) controllers from industrial systems to coordinate multiple training objectives in graph machine learning, allocating computational capacity to one objective at a time rather than blending gradients at each step. The method achieved average ranks of 1.4, 1.9, and 1.8 across node classification, link prediction, and node clustering tasks on nine graph benchmarks, outperforming existing multitask methods including CAGrad and AutoSSL. This approach eliminates instantaneous gradient conflicts between objectives and could extend to other multitask learning scenarios like language model fine-tuning where competing training objectives cause performance degradation.
Why it matters
Instead of compromising among parameter updates dictated by different training objectives, ControlG allocates computational capacity to objectives sequentially and dynamically.
