Learning with not Enough Data Part 3: Data Generation

Lilian Weng · 4 years ago

The article discusses two approaches for generating synthetic training data when real data is scarce: data augmentation through transformations of existing samples, and generating new data using pretrained models like large language models. Data augmentation modifies input format while preserving semantic meaning, and few-shot prompting enables language models to learn from limited examples without additional training. This addresses the challenge of training machine learning models with insufficient real-world data.