Bootstrapping Labels via ___ Supervision & Human-In-The-Loop

Eugene Yan · 5 years ago · 25

The article discusses methods for obtaining training labels when labeled datasets are unavailable, covering semi-supervised learning (using pseudo-labels from model predictions), active learning (selecting uncertain or informative samples for human annotation), and weak supervision (combining multiple lower-quality label sources). DoorDash used active learning with human-in-the-loop for menu tagging, Facebook applied similarity search filtering to reduce annotation overhead to 5% of samples, and Google used Snorkel DryBell with labeling functions to match performance that would have required 80,000 hand-labeled samples. Organizations can reduce labeling costs and scale annotation workflows by combining these approaches with well-defined labeling guidelines and creative use of existing user data or weak signal sources.