Learning with not Enough Data Part 1: Semi-Supervised Learning

Lilian Weng · 4 years ago

Semi-supervised learning methods train models on both labeled and unlabeled data by combining supervised loss with unsupervised loss that encourages consistent predictions across different data perturbations. The approach includes techniques like Π-model, Temporal Ensembling, Mean Teacher, Virtual Adversarial Training, and Interpolation Consistency Training, each using different strategies to leverage unlabeled data. This article represents part 1 of a series on learning with limited labeled data and focuses on consistency regularization methods that have shown effectiveness primarily on vision tasks.