Personality Assessment on Spanish and Basque Texts using In-Context Learning Techniques

Resumen

This study assesses the performance of Llama3 generative large language models (8B and 70B) in predicting Big Five personality traits from Spanish and Basque texts. Various in-context learning approaches, including zero-shot, few-shot, and Chain-of-Thought (CoT) prompting, as well as instruction fine-tuning, were evaluated on two datasets built on texts from different sources, Essays and PAN-15 (with a Basque subset translated for this work). Results show that Llama3 performs poorly in Basque, with in-context learning strategies failing to exceed the random baseline, except for a slight improvement with CoT on the 70B model. Fine-tuning the 8B model provides only marginal gains. Performance in Spanish is better but remains modest, with one-shot prompting and fine-tuning offering slight improvements in the case of the smaller model. Finally, in the case of Spanish, all in-context learning techniques surpass zero-shot when using the 70B model.