Language Discrimination Improves Linguistic Learning in Multilingual Speech Models

Apple Machine Learning Research · 1 week ago · 18

Multilingual HuBERT-style self-supervised speech models improved linguistic learning when pretraining was made to better discriminate between languages. Phone-ABX error dropped from 11.6% in the bilingual baseline to 10.4% (monolingual: 10.8%) while lexical (sWUGGY) and prosodic performance also increased. The gains were largest when language discrimination was introduced in the first training iteration, whereas later or repeated use caused more language-wise segregation.