Systematic Review of Emotion Detection with Computer Vision and Deep Learning

Pereira, Rafael; Mendes, Carla; Ribeiro, José; Ribeiro, Roberto; Miragaia, Rolando; Rodrigues, Nuno; Costa, Nuno; Pereira, António

http://hdl.handle.net/10400.8/9898

Use this identifier to reference this record.

Name:	Description:	Size:	Format:
sensors-24-03484-v2.pdf		615.79 KB	Adobe PDF	Download

Send Feedback

Authors

Abstract(s)

Emotion recognition has become increasingly important in the field of Deep Learning (DL) and computer vision due to its broad applicability by using human–computer interaction (HCI) in areas such as psychology, healthcare, and entertainment. In this paper, we conduct a systematic review of facial and pose emotion recognition using DL and computer vision, analyzing and evaluating 77 papers from different sources under Preferred Reporting Items for Systematic Reviews and MetaAnalyses (PRISMA) guidelines. Our review covers several topics, including the scope and purpose of the studies, the methods employed, and the used datasets. The scope of this work is to conduct a systematic review of facial and pose emotion recognition using DL methods and computer vision. The studies were categorized based on a proposed taxonomy that describes the type of expressions used for emotion detection, the testing environment, the currently relevant DL methods, and the datasets used. The taxonomy of methods in our review includes Convolutional Neural Network (CNN), Faster Region-based Convolutional Neural Network (R-CNN), Vision Transformer (ViT), and “Other NNs”, which are the most commonly used models in the analyzed studies, indicating their trendiness in the field. Hybrid and augmented models are not explicitly categorized within this taxonomy, but they are still important to the field. This review offers an understanding of state-of-the-art computer vision algorithms and datasets for emotion recognition through facial expressions and body poses, allowing researchers to understand its fundamental components and trends.

Description

Funding: This work was supported by national funds through the Portuguese Foundation for Science and Technology (FCT), I.P., under the project UIDB/04524/2020.

Keywords

Emotion recognition Computer vision Deep learning Systematic review Emotion detection

URI

http://hdl.handle.net/10400.8/9898

Citation

Pereira, R.; Mendes, C.; Ribeiro, J.; Ribeiro, R.; Miragaia, R.; Rodrigues, N.; Costa, N.; Pereira, A. Systematic Review of Emotion Detection with Computer Vision and Deep Learning. Sensors 2024, 24(11), 3484. http://doi.org/10.3390/s24113484