Shape, Symmetries, and Structure: The Changing Role of Mathematics in Machine Learning Research
The Gradient Henry Kvinge
Machine learning research has shifted from theory-driven mathematical approaches to empirical, compute-intensive methods that achieve breakthroughs unpredicted by existing theory, prompting discussion about mathematics' diminished role in the field. Rather than disappearing, mathematics is evolving to serve new purposes such as post-hoc explanation of phenomena and matching architectures to data symmetries, with pure mathematical fields like topology and geometry now joining traditional applied mathematics. Mathematical tools like intrinsic dimension and curvature are being applied to analyze neural network weights and activations, enabling researchers to characterize high-dimensional model properties and detect adversarial examples or hallucinations.
Why it matters
What is the Role of Mathematics in Modern Machine Learning?The past decade has witnessed a shift in how progress is made in machine learning. Research involving carefully designed and mathematically principled architectures result in only marginal improvements while compute-intensive and engineering-first efforts that scale to ever larger training sets
