The Role of Emotions

Investigating Communicative Roles in Models and Data for Emotion Recognition

Abstract

Emotion recognition is a well-studied Natural Language Understanding task. However, datasets for this task are annotated in different ways: some reflect the emotions of the original speaker or author, while others rely on observer judgments from third-party annotators. These differing roles raise questions about how language models respond to dataset annotation roles and how prompt style and role influence model behavior. In this work, we conduct extensive experiments that vary prompt style and prompt role as well as model role, using datasets labeled by speakers or by observers from a recently introduced emotion benchmark. We propose a speaker-observer framing for model evaluation, distinguishing decoder-based models (e.g., Llama-3.1) as speaker models, and encoder-based models (e.g., RoBERTa) as observer models, and evaluate whether alignment between model behavior, prompt framing, and dataset annotation role improves performance. Preliminary results provide mixed evidence for such role alignment effects, suggesting that the interaction between prompt, model, and annotation role is nuanced and task-dependent, motivating more role-aware evaluation practices for language models.