Shape, Symmetries, and Structure: The Changing Role of Mathematics in Machine Learning Research

The Gradient · 1 year ago

Machine learning research has shifted from theory-driven mathematical approaches to empirical, compute-intensive methods that achieve breakthroughs unpredicted by existing theory, prompting discussion about mathematics' diminished role in the field. Rather than disappearing, mathematics is evolving to serve new purposes such as post-hoc explanation of phenomena and matching architectures to data symmetries, with pure mathematical fields like topology and geometry now joining traditional applied mathematics. Mathematical tools like intrinsic dimension and curvature are being applied to analyze neural network weights and activations, enabling researchers to characterize high-dimensional model properties and detect adversarial examples or hallucinations.