Towards EEG-based Emotion Recognition During Video Viewing

Neural-Congruency Explains User’s Emotion experienced in Music Videos

Abstract

Emotions affect our decisions, experiences, preferences, and perceptions. Understanding the neural underpinning of human emotions is a fundamental goal of neuroscience research. Moreover, EEG-based emotion recognition is a key component towards the development of affective-aware intelligent systems. However, characterizing the neural basis of emotions elicited during video viewing has been proven a challenging task. In this paper, we propose a novel machine-learning approach to isolate neural components in EEG signals that are informative of the affective content of emotionally-loaded videos. Based on these components, we define a set of neural metrics and evaluate them as potential indicators of the overall emotional content of each video. We demonstrate the predictive power of the proposed metrics, on the DEAP benchmark dataset for EEG-based emotion recognition. Our results provide novel empirical evidence that the neural components extracted by our method can serve as an informative metric in EEG-based emotion recognition during video viewing and achieving a 4-fold increase in predictive power compared to traditional frequency-based metrics. Moreover, each extracted component is associated with a spatial and a temporal profile, that allows researchers to inspect and interpret the spatiotemporal origins of the underlying neural activity. Thus, our method a framework that facilitates the study of neural correlates of emotion during video viewing.