Improving language understanding with unsupervised learning
OpenAI 8 years ago 22
Researchers combined transformer models with unsupervised pre-training to achieve top results across multiple language understanding tasks. The system was tested on a diverse suite of benchmarks and released publicly. The findings suggest that unsupervised pre-training paired with supervised learning is effective, potentially encouraging further research at larger scales.