When Unlearning Is Free: Leveraging Low Influence Points to Reduce Computational Costs
Apple ML Research
Researchers developed an unlearning framework that identifies training data points with negligible influence on model outputs and removes them before the unlearning process. The method achieves approximately 50% computational savings on real-world examples by reducing dataset size prior to unlearning. This enables faster and more efficient removal of specific data from trained models while maintaining privacy compliance.
Why it matters
As concerns around data privacy in machine learning grow, the ability to unlearn—or remove—specific data points from trained models becomes increasingly important. While state-of-the-art unlearning methods have emerged in response, they typically treat all points in the forget set equally. In this work, we challenge this approach by asking: do points that have a negligible impact on the model’s learning need to be removed? Through a comparative analysis of influence functions across language and vision tasks, we identify subsets of training data with negligible impact on model outputs…
