Fine-Tune XLSR-Wav2Vec2 for low-resource ASR with 🤗 Transformers

Hugging Face · 4 years ago · 23

XLS-R, a multilingual speech recognition model trained on 500,000 hours of audio across 128 languages, can be fine-tuned for automatic speech recognition tasks in low-resource languages. The model comes in three sizes ranging from 300 million to 2 billion parameters and uses masked feature vector learning similar to BERT's pretraining approach. Fine-tuning XLS-R on labeled datasets like Common Voice requires adding a linear classification layer on top of the pretrained network and applying Connectionist Temporal Classification to map speech representations to text transcriptions.