UNED-RAG ES: A Spanish Dataset for Evaluating RAG Generation under Context Noise

Resumen

We present UNED-RAG ES, an annotated Spanish resource designed to evaluate the generation stage of Retrieval-Augmented Generation (RAG) independently of retrieval. The dataset contains 261 test units in the academic and administrative domain, each with a natural-language question, two reference answers, and supporting documents with explicitly marked evidence. It is intended as a base resource from which benchmarks can be instantiated; we derive two controlled benchmarks that vary (i) the amount of semantic noise and (ii) the position of supporting evidence within the context. We validate the resource by benchmarking commercial and open-weight large language models using reference-based similarity metrics that compare model outputs with the annotated answers. Results show an overall degradation trend as noise increases and model-dependent sensitivity to evidence position, supporting UNED-RAG ES as a practical resource for stress-testing Spanish RAG generation under imperfect contexts.