Hybrid Annotation Framework for Spanish Subjectivity Detection

Resumen

Subjectivity Detection (SD) is a fundamental NLP task that distinguishes between factual content and opinion. Its analysis provides profound linguistic knowledge, useful for tasks such as disinformation detection. Our objective is to develop and validate a hybrid annotation framework for SD in Spanish. Our methodology includes: (1) a pilot study comparing human annotation against Machine Learning (ML) and Large Language Models (LLMs); (2) an error analysis to refine the framework; and (3) the application of the consolidated framework. Results show ML achieving higher precision, but overlooking complex structures (e.g., subjunctive). The consolidated hybrid methodology is tested in three annotation setups (human [A1], unrevised automatic [A2], consolidated semi-automatic [A3]). A3 achieves the best balance between quality and time consumption (only 19% of the manual annotation effort).