Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions
Apple Machine Learning Research
Cross-lingual knowledge transfer is presented as essential for training multilingual language models on low-data languages, because key knowledge for downstream tasks must come mainly from a high-resource language when target data is scarce. No specific number, date, or benchmark appears in the provided excerpt. The approach described shifts to using lexical interventions to move knowledge across languages under data constraints.
Why it matters
Cross-lingual knowledge transfer is critical for building high-performing multilingual language models for languages with insufficient training data. When target language data is scarce, the knowledge required for many downstream tasks involving scientific reasoning, commonsense inference, and world knowledge must be acquired primarily from the high-resource language, making effective knowledge transfer essential. Existing methods for improving such cross-lingual knowledge transfer require large amounts of parallel data, translation systems, auxiliary models, or additional training stages that…