From Rule-Based to LLMs: A Performance and Variability Analysis of Galician Machine Translation Models

Resumen

This paper evaluates machine translation (MT) for English–Galician, Spanish–Galician, and Portuguese–Galician pairs, with the aim of identifying the most effective models for these language pairs in the general domain. The evaluation encompasses a range of factors, including model quality, performance variance and size. The assessment involves the evaluation of different open-source systems. The results obtained identify that, for Spanish–Galician, both a Rule-Based System and a bilingual Neural Machine Translation model outperform larger multilingual models and LLMs. However, for more distant language pairs, multilingual models demonstrate superior performance. The study underscores the necessity for further research in Portuguese–Galician pair.