Overview of DIPROMATS 2024: Detection, Characterization and Tracking of Propaganda in Messages from Diplomats and Authorities of World Powers

Pablo Moral, Jesús M. Fraile, Guillermo Marco, Anselmo Peñas, Julio Gonzalo

Resumen


This paper summarizes the findings of DIPROMATS 2024, a challenge included at the Iberian Languages Evaluation Forum (IberLEF). This second edition introduces a refined typology of techniques and a more balanced dataset for propaganda detection, alongside a new task focused on identifying strategic narratives. The dataset for the first task includes 12,012 annotated tweets in English and 9,501 in Spanish, posted by authorities from China, Russia, the United States, and the European Union. Participants tackled three subtasks in each language: binary classification to detect propagandistic tweets, clustering tweets into three propaganda categories, and fine-grained categorization using seven techniques. The second task presents a multi-class, multi-label classification challenge where systems identify which predefined narratives (associated with each international actor) tweets belong to. This task is supported by narrative descriptions and example tweets in English and Spanish, using few-shot learning techniques. 40 runs from nine different teams were evaluated.

Texto completo:

PDF