Emotion Recognition with Multi Physiological Signals: A Deep Learning Approach

Abstract

Human emotions, a complex interplay of psychological and physiological signals are a critical aspect of human interaction and well-being. Emotion recognition models in general capture human behaviour via facial features, voice/speech, and physiological signals, and evaluate and predict emotional states. The physiological signals, like brainwave, heart rate, eye movement, or galvanic skin response are the major cause of emotional changes. The combination of these signals contributes to the emotion change. Thus, effective models with combined physiological signals will serve better solutions compared to other modalities in practice. The rapid and precise recognition of emotions remains a challenge due to not enough combined physiological datasets. The promising characteristics of deep learning will help in achieving efficient emotional recognition models in place which use physiological signals as the predominant concern. The paper aims to propose a framework that utilizes Convolutional Neural Networks (CNN) with feature engineering to process combined physiological signals enabling the model to discern relevant emotional patterns while filtering out noise. To improve the consistency and accuracy of the prediction of emotions Linear Regression (LR) is used and Random Forest (RF) was employed to reduce the overfitting and noise problem. The efficacy of the proposed model was rigorously evaluated using standard performance metrics which are precision, recall, F1 score and accuracy. The proposed model, CNN+LR+RF with feature optimization model performed well with 61% accuracy compared to other proposed models.