Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions

Summary

Cross-lingual knowledge transfer is critical for multilingual models with scarce target-language data. Existing methods require parallel data, translation systems, or extra training stages. The paper proposes LINK, a data-level intervention that substitutes words in high-resource (English) pretraining data with translations using bilingual vocabularies, requiring no extra training. Evaluations on eight languages and five model sizes show notable downstream improvements and up to a 2x training speedup to reach equivalent performance.