Knowledge Transfer from High-Resource to Low-Resource Languages for Code LLMs (2023) (arxiv.org)

Researchers introduced MultiPL-T, a semi-synthetic data pipeline that boosts Code LLM performance on low-resource programming languages (Julia, Lua, OCaml, R, Racket) by translating high-quality examples from high-resource languages. The method first uses a Code LLM to synthesize unit tests for commented high-resource code, filters out faulty tests and low-coverage examples, then translates validated Python solutions into the target low-resource language using a Code LLM and re-runs the tests to validate translations. This produced tens of thousands of validated training items used to fine-tune open models (StarCoderBase and Code Llama) without contaminating benchmarks or violating licenses by leveraging The Stack and open tooling. Significance: MultiPL-T offers a practical, scalable way to extend Code LLM capabilities to underrepresented languages faster and cheaper than simply training longer or collecting native corpora. On the MultiPL-E benchmark the fine-tuned models outperform other open Code LLMs, demonstrating that test-driven, model-assisted translation can create high-quality training data. Key technical implications include a reusable, language-agnostic pipeline for dataset synthesis, stronger experimental reproducibility and license compliance because of open-source bases, and a dependency on the quality of synthesized tests/translations—mitigated but not eliminated by test validation.