MLLM-as-a-Judge for Financial Document Image Machine Translation

Resumen

Document Image Machine Translation (DIMT) enables direct translation from visuals, but traditional Machine Translation (MT) metrics ignore the source image during evaluation. This paper explores an end-to-end (E2E) DIMT approach using Gemma 4 on financial reports from IBEX 35 companies. We propose a Multimodal LLM-as-a-judge (MLLMJ) to provide a grounded and explainable evaluation that considers both visual and textual information. It follows a rubric to assign scores and provides both a justification and an internal thinking process leading to the assigned score. Additionally, we estimate the system's potential upper bound through Oracle-Guided Self-Refinement (OGSR), using the judge’s feedback to rectify initial predictions. Our study evaluates various configurations, including zero-shot and fine-tuning scenarios across different model sizes. By focusing on image-aware evaluation, this work attempts to offer a framework that provides reasoning cues for improving E2E DIMT systems and enables future knowledge distillation.