Improving language understanding with unsupervised learning

OpenAI Blog · 8 years ago

Researchers combined transformer models with unsupervised pre-training to achieve top results across multiple language understanding tasks. The system was tested on a diverse suite of benchmarks and released publicly. The findings suggest that unsupervised pre-training paired with supervised learning is effective, potentially encouraging further research at larger scales.