ArXiv Publishes: Automating Data Pipelines and Reproducing Baselines for Low-Resource Multilingual Translation

Summary:ArXiv has published a study focused on optimizing low-resource multilingual translation systems, specifically enhancing a small neural machine translation (NMT) system for Sicilian. The research introduces automated data pipeline construction, parallel text extraction and filtering, and language-agnostic sentence alignment to improve translation quality. Experiments show that using a pretrained NLLB model with low-rank fine-tuning and reverse training outperforms training from scratch, particula

---

Background and Motivation

In recent years, multilingual translation systems have faced challenges in handling low-resource languages due to insufficient data quality and quantity. Sicilian, as a low-resource language, presents particular difficulties in achieving high-quality translations. This study aims to enhance the performance of a Sicilian translation system by optimizing data pipelines and model training strategies.

Technical Mechanism Analysis

1. Automated Data Pipeline and Parallel Text Extraction

   • The research team redeveloped the small NMT system customized for Sicilian by Wdowiak and rebuilt the extraction of parallel text from the bilingual journal Arba Sicula's PDFs using maintained software.
   • A language-agnostic sentence encoder was used for sentence alignment, with thresholds tuned against existing hand alignments and filtered for typical errors in automatic alignment.

2. Model Training and Optimization

   • The tokenization and desinence-biased subword segmentation of the original system were ported to a Perl-free implementation.
   • The team compared Transformer models trained from scratch with a pretrained NLLB model adapted by low-rank fine-tuning.
   • A reverse training strategy was employed, starting with forward-scored back-translations of web Sicilian followed by training on curated literary text.

3. Experimental Results and Evaluation

   • The pretrained model outperformed the model trained from scratch in all directions, particularly for translations into Sicilian.
   • The multilingual adapter degraded performance in directions it was not trained on, limiting its use as a general multilingual model.

Engineering Trade-offs and Performance

• Strengths:
  • Automated data pipelines and optimized training strategies significantly improve translation quality for low-resource languages.
  • Pretrained models combined with reverse training strategies perform well in resource-constrained environments.
• Weaknesses:
  • The multilingual adapter's performance decreases in untrained directions, limiting its versatility.
  • Automated alignment methods still contain errors and require further optimization.

Developer Deployment and Recommendations

• Recommendations:
  • For low-resource language translation tasks, it is recommended to use pretrained models with low-rank fine-tuning and reverse training strategies.
  • The automated data pipeline and parallel text extraction methods can be extended to other low-resource languages to improve data quality and quantity.
  • The use of multilingual adapters should be carefully evaluated based on specific task requirements.

---

Source: ArXiv NLP/LLM (cs.CL) (2026-10-24)

— END —

Tags: #Multilingual Translation Follow #Low-Resource Languages Follow #NMT Follow #Pretrained Models Follow #Reverse Training Follow