Loading...
Publication
EMPATHETIC CONVERSATIONAL AGENT FROM BODY EMOTION DETECTION WITH AI
| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 2.98 MB | Adobe PDF |
Authors
Abstract(s)
A interação entre humano-computador está constantemente a evoluir com o surgimento
de novos agentes conversacionais promissores. No entanto, estes agentes geralmente não
possuem a capacidade de compreender ou reconhecer as emoções dos seus utilizadores.
No contexto do reconhecimento de emoções, os sistemas mais comuns na investigação
científica baseiam-se em expressões faciais e análise de voz. Apesar dos avanços
significativos no reconhecimento de emoções, algumas formas de expressão emocional
permanecem pouco exploradas, como o reconhecimento de emoções através de gestos
corporais, representados por sequências de imagens que capturam expressões corporais.
Esta tese propõe uma arquitetura para construir um agente conversacional empático
utilizando o reconhecimento de emoções através de gestos corporais. Esta estratégia
faz parte de uma arquitetura geral de HCI que combina informações emocionais
provenientes de múltiplas fontes, incluindo expressões faciais, gestos corporais, análise
de voz e análise de sentimento, integrando-as através de um modelo de fusão. O
design desta arquitetura geral é baseado numa análise extensa de artigos científicos
sobre o reconhecimento de emoções. Nesta tese foram realizados vários experimentos
com diferentes modelos de AI para realizar a deteção de emoções com base em gestos
corporais, utilizando o conjunto de dados VGAF_EmotiW, que contém gestos corporais
em diferentes contextos. O modelo com melhor desempenho foi o PoseGRU, que atingiu
taxas de precisão superiores a 50%.
Quando investigadores ou programadores pretendem implementar um sistema
semelhante ao proposto nesta tese, muitas vezes precisam começar do zero, sem saber
por onde começar ou quais passos seguir. No momento da escrita desta tese, e com
base numa revisão de literatura sistemática publicada presente nesta tese, não existem
guidelines disponíveis que auxiliem na criação de uma solução como a que é proposta.
Esta falta de orientação estruturada pode representar um desafio para a implementação
eficaz de sistemas de de conversação empáticos que fazem uso do reconhecimento de
gestos corporais.
Para abordar esta lacuna, a tese define um conjunto de procedimentos para a
implementação da solução proposta. Estes procedimentos, apresentados sob a forma
de guidelines, oferecem uma estrutura passo a passo para ajudar investigadores e
programadores a construir a solução proposta ou sistemas semelhantes, detalhando
claramente as etapas necessárias para a implementação.
As guidelines são também demonstradas detalhadamente e aplicadas a três casos de uso diferentes: atendimento ao cliente, educação e aprendizagem online, e aplicações
para a saúde mental. Ao seguir estas diretrizes, o sistema pretende melhorar a interação
entre o humano e o computador ao adaptar-se aos estados emocionais dos utilizadores,
o que, na prática, pode melhorar a comunicação e a ligação entre ambos. Por exemplo,
em ambientes empresariais e/ou quando aplicadas a utilizadores idosos, a solução
pode aumentar a satisfação, melhorar a qualidade das interações e contribuir para o
sucesso geral do negócio. Além disso, esta tese aborda seis questões de investigação que
fornecem respostas a questões relevantes e pertinentes no campo do reconhecimento de
emoções e da interação humano-computador empática.
Human-computer interaction is rapidly evolving with the emergence of promising conversational agents. However, these agents generally lack the ability to understand or recognise the emotions of their users. In the context of emotion recognition, the most common systems in scientific research rely on facial expressions and voice analysis. Despite significant advances in emotion recognition, certain forms of emotional expression remain underexplored, such as the recognition of emotions through body gestures, represented by sequences of images capturing body expressions. This thesis proposes an architecture for building an empathetic conversational agent using emotion recognition through body gestures. This strategy is part of a general HCI architecture that combines emotional information from multiple sources, including facial expressions, body gestures, voice analysis, and sentiment analysis, and integrates them through a fusion model. The design of this general architecture is based on an extensive analysis of scientific articles on emotion recognition. In this thesis, a number of experiments were conducted with different AI models to perform emotion detection based on body gestures, using the VGAF_EmotiW dataset, which contains body gestures in different contexts. The best performing modelwas PoseGRU, which achieved accuracy rates of just over 50%. When researchers or developers want to implement a system similar to the one proposed in this thesis, they often have to start from scratch, sometimes without knowing where to start or what steps to follow. At the time of writing, based on a published systematic literature review of the existing research documented in this thesis, there are no guidelines available to assist in the creation of a solution such as the one proposed here. This lack of structured guidance can pose a challenge to the effective implementation of emotional conversation agents systems that rely on body gesture recognition. To address this gap, the thesis defines a set of procedures for implementing the proposed solution. These procedures, outlined as guidelines, provide a step-by-step framework to help researchers and developers build the proposed solution or similar systems by clearly detailing the necessary stages of implementation. The guidelines are also demonstrated step-by-step and applied to three different use cases: Customer service, education and e-learning, and mental health applications. By following these guidelines, the system aims to improve human-computer interaction by adapting to users’ emotional states, which in practice can improve communication and engagement. For example, in business environments or when applied to elderly users, the solution can increase satisfaction, improve the quality of interactions and contribute to overall business success. The guidelines execution for these use cases also paves the way for the guidelines to be explored in other areas. In addition, this thesis addresses six research questions that provide answers to relevant and pertinent issues in the field of emotion recognition and empathetic human-computer interaction.
Human-computer interaction is rapidly evolving with the emergence of promising conversational agents. However, these agents generally lack the ability to understand or recognise the emotions of their users. In the context of emotion recognition, the most common systems in scientific research rely on facial expressions and voice analysis. Despite significant advances in emotion recognition, certain forms of emotional expression remain underexplored, such as the recognition of emotions through body gestures, represented by sequences of images capturing body expressions. This thesis proposes an architecture for building an empathetic conversational agent using emotion recognition through body gestures. This strategy is part of a general HCI architecture that combines emotional information from multiple sources, including facial expressions, body gestures, voice analysis, and sentiment analysis, and integrates them through a fusion model. The design of this general architecture is based on an extensive analysis of scientific articles on emotion recognition. In this thesis, a number of experiments were conducted with different AI models to perform emotion detection based on body gestures, using the VGAF_EmotiW dataset, which contains body gestures in different contexts. The best performing modelwas PoseGRU, which achieved accuracy rates of just over 50%. When researchers or developers want to implement a system similar to the one proposed in this thesis, they often have to start from scratch, sometimes without knowing where to start or what steps to follow. At the time of writing, based on a published systematic literature review of the existing research documented in this thesis, there are no guidelines available to assist in the creation of a solution such as the one proposed here. This lack of structured guidance can pose a challenge to the effective implementation of emotional conversation agents systems that rely on body gesture recognition. To address this gap, the thesis defines a set of procedures for implementing the proposed solution. These procedures, outlined as guidelines, provide a step-by-step framework to help researchers and developers build the proposed solution or similar systems by clearly detailing the necessary stages of implementation. The guidelines are also demonstrated step-by-step and applied to three different use cases: Customer service, education and e-learning, and mental health applications. By following these guidelines, the system aims to improve human-computer interaction by adapting to users’ emotional states, which in practice can improve communication and engagement. For example, in business environments or when applied to elderly users, the solution can increase satisfaction, improve the quality of interactions and contribute to overall business success. The guidelines execution for these use cases also paves the way for the guidelines to be explored in other areas. In addition, this thesis addresses six research questions that provide answers to relevant and pertinent issues in the field of emotion recognition and empathetic human-computer interaction.
Description
Keywords
Deteção de emoções por gestos corporais Agentes conversacionais empáticos Aprendizagem profunda Interação humano-computador (HCI)