Learning with not Enough Data Part 1: Semi-Supervised Learning
Lil'Log
Semi-supervised learning methods train models on both labeled and unlabeled data by combining supervised loss with unsupervised loss that encourages consistent predictions across different data perturbations. The approach includes techniques like Π-model, Temporal Ensembling, Mean Teacher, Virtual Adversarial Training, and Interpolation Consistency Training, each using different strategies to leverage unlabeled data. This article represents part 1 of a series on learning with limited labeled data and focuses on consistency regularization methods that have shown effectiveness primarily on vision tasks.
Why it matters
When facing a limited amount of labeled data for supervised learning tasks, four approaches are commonly discussed.