journal article·2023-01-15Open access

Prediction of Emotional Empathy in Intelligent Agents to Facilitate Precise Social Interaction

Saad Awadh Alanazi, Maryam Shabbir, Nasser Alshammari, Madallah Alruwaili, Iftikhar Hussain, Fahad Ahmad

Applied Sciences· Volume 13· Issue 2

Abstract

The research area falls under the umbrella of affective computing and seeks to introduce intelligent agents by simulating emotions artificially and encouraging empathetic behavior in them, to foster emotional empathy in intelligent agents with the overarching objective of improving their autonomy. Raising the emotional empathy of intelligent agents to boost their autonomic behavior can increase their independence and adaptability in a socially dynamic context. As emotional intelligence is a subset of social intelligence, it is essential for successful social interaction and relationships. The purpose of this research is to develop an embedded method for analyzing empathic behavior in a socially dynamic situation. A model is proposed for inducing emotional intelligence through a deep learning technique, employing multimodal emotional cues, and triggering appropriate empathetic responses as output. There are 18 categories of emotional behavior, and each one is strongly influenced by multimodal cues such as voice, facial, and other sensory inputs. Due to the changing social context, it is difficult to classify emotional behavior and make predictions based on modest changes in multimodal cues. Robust approaches must be used to be sensitive to these minor changes. Because a one-dimensional convolutional neural network takes advantage of feature localization to minimize the parameters, it is more efficient in this exploration. The study’s findings indicate that the proposed method outperforms other popular ML approaches with a maximum accuracy level of 98.98 percent when compared to currently used methods.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Emotion and Mood Recognition · Social Robot Interaction and HRI · Technology and Human Factors in Education and Health

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Computer Science, College of Computer and Information Sciences, Jouf University, Sakaka 72341, Saudi Arabia

School of Professional Advancement, University of Management and Technology, Lahore 54700, Pakistan

Department of Computer Engineering and Networks, College of Computer and Information Sciences, Jouf University, Sakaka 72341, Saudi Arabia

Center for Sustainable Road Freight and Business Management, Heriot-Watt University, Edinburgh EH14 4AS, UK

Delta3T, Lahore 54700, Pakistan

Funding

Deanship of Scientific Research at Jouf University

References (55)

Show source-provided references
1. Artificial intelligence: A powerful paradigm for scientific research
2. Thakur, N., and Han, C.Y. (2021). Indoor Localization for Personalized Ambient Assisted Living of Multiple Users in Multi-Floor Smart Environments. Big Data Cogn. Comput., 5.10.3390/bdcc5030042 ↗
3. Baluprithviraj, K., Bharathi, K., Chendhuran, S., and Lokeshwaran, P. (2021, January 25–27). Artificial intelligence based smart door with face mask detection. Proceedings of the 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), Madukkarai, India.10.1109/ICAIS50930.2021.9395807 ↗
4. Empathy: Its ultimate and proximate bases10.1017/S0140525X02000018 ↗
5. Millon, I.T., and Lerner, M.J. (2003). Altruism and Prosocial Behavior, John Wiley & Sons, Inc.
6. Tao, J., and Tan, T. (2005, January 22–24). Affective computing: A review. Proceedings of the International Conference on Affective computing and intelligent interaction, Beijing, China.10.1007/11573548 ↗
7. Pantic, M., Caridakis, G., André, E., Kim, J., Karpouzis, K., and Kollias, S. (2011). Multimodal emotion recognition from low-level cues. Emotion-Oriented Systems, Springer.10.1007/978-3-642-15184-2_8 ↗
8. Savery, R., and Weinberg, G. (September, January 31). A survey of robotics and emotion: Classifications and models of emotional interaction. Proceedings of the 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Naples, Italy.10.1109/RO-MAN47096.2020.9223536 ↗
9. Biondi, G., Franzoni, V., and Poggioni, V. (2017, January 3–6). A deep learning semantic approach to emotion recognition using the IBM watson bluemix alchemy language. Proceedings of the International Conference on Computational Science and Its Applications, Trieste, Italy.10.1007/978-3-319-62398-6_51 ↗
10. Computational sustainability and the PHESS platform: Using affective computing as social indicators10.1016/j.future.2018.10.006 ↗
11. Fan, L., Scheutz, M., Lohani, M., McCoy, M., and Stokes, C. (2017, January 27–30). Do we need emotionally intelligent artificial agents? First results of human perceptions of emotional intelligence in humans compared to robots. Proceedings of the International Conference on Intelligent Virtual Agents, Stockholm, Sweden.10.1007/978-3-319-67401-8_15 ↗
12. Huang, Y., Fei, T., Kwan, M.-P., Kang, Y., Li, J., Li, Y., Li, X., and Bian, M. (2020). GIS-Based Emotional Computing: A Review of Quantitative Approaches to Measure the Emotion Layer of Human–Environment Relationships. ISPRS Int. J. Geo-Inf., 9.10.3390/ijgi9090551 ↗
13. Empathy for artificial agents10.1007/s12369-014-0260-0 ↗
14. The impact of using different annotation schemes on named entity recognition10.1016/j.eij.2020.10.004 ↗
15. Affective computing10.1007/BF01238028 ↗
16. Development of artificial empathy10.1016/j.neures.2014.12.002 ↗
17. Intelligent humanoid robots expressing artificial humanlike empathy in nursing situations10.1111/nup.12318 ↗
18. Nguyen, H., and Masthoff, J. (2009, January 4–6). Designing empathic computers: The effect of multimodal empathic feedback using animated agent. Proceedings of the 4th international conference on persuasive technology, Amsterdam, The Netherlands.10.1145/1541948.1541958 ↗
19. Empathy in virtual agents and robots: A survey10.1145/2912150 ↗
20. Two facets of affective empathy: Concern and distress have opposite relationships to emotion recognition10.1080/02699931.2020.1724893 ↗
21. Facial mimicry, empathy, and emotion recognition: A meta-analysis of correlations10.1080/02699931.2020.1815655 ↗
22. Luna-Jiménez, C., Griol, D., Callejas, Z., Kleinlein, R., Montero, J.M., and Fernández-Martínez, F. (2021). Multimodal Emotion Recognition on RAVDESS Dataset Using Transfer Learning. Sensors, 21.10.3390/s21227665 ↗
23. Liu, W., Lee, K.-P., Gray, C.M., Toombs, A.L., Chen, K.-H., and Leifer, L. (2021). Transdisciplinary teaching and learning in UX design: A program review and AR case studies. Appl. Sci., 11.10.3390/app112210648 ↗
24. Desmet, P., and Fokkinga, S. (2020). Beyond Maslow’s pyramid: Introducing a typology of thirteen fundamental needs for human-centered design. Multimodal Technol. Interact., 4.10.3390/mti4030038 ↗
25. Social touch gesture recognition using convolutional neural network10.1155/2018/6973103 ↗
26. Dzedzickis, A., Kaklauskas, A., and Bucinskas, V. (2020). Human emotion recognition: Review of sensors and methods. Sensors, 20.10.3390/s20030592 ↗
27. Facial landmark detection: A literature survey10.1007/s11263-018-1097-z ↗
28. Chandra, R. (2017, January 4–9). Towards an affective computational model for machine consciousness. Proceedings of the International Conference on Neural Information Processing, Long Beach, CA, USA.10.1007/978-3-319-70139-4_91 ↗
29. Emotional affordances for human–robot interaction10.1177/1059712316668238 ↗
30. Mohammadi, H.B., Xirakia, N., Abawi, F., Barykina, I., Chandran, K., Nair, G., Nguyen, C., Speck, D., Alpay, T., and Griffiths, S. (June, January 29). Designing a personality-driven robot for a human-robot interaction scenario. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, Canada.10.1109/ICRA.2019.8793770 ↗
31. Nocentini, O., Fiorini, L., Acerbi, G., Sorrentino, A., Mancioppi, G., and Cavallo, F. (2019). A survey of behavioral models for social robots. Robotics, 8.10.20944/preprints201905.0251.v1 ↗
32. Deliberative safety for industrial intelligent human–robot collaboration: Regulatory challenges and solutions for taking the next step towards industry 4.010.1016/j.rcim.2022.102386 ↗
33. From face to face: The contribution of facial mimicry to cognitive and emotional empathy10.1080/02699931.2019.1596068 ↗
34. Franzoni, V., Biondi, G., Perri, D., and Gervasi, O. (2020). Enhancing Mouth-Based Emotion Recognition Using Transfer Learning. Sensors, 20.10.20944/preprints202007.0379.v1 ↗
35. Robots and emotion: A survey of trends, classifications, and forms of interaction10.1080/01691864.2021.1957014 ↗
36. Lugrin, B., Pelachaud, C., and Traum, D. (2022). The Handbook on Socially Interactive Agents: 20 Years of Research on Embodied Conversational Agents, Intelligent Virtual Agents, and Social Robotics, Volume 2: Interactivity, Platforms, Application, Association for Computing Machinery.10.1145/3563659 ↗
37. Cummins, N., Provost, E., Sethu, V., Epps, J., Busso, C., and Narayanan, S. (2019). The ambiguous world of emotion representation. arXiv.
38. Machine Learning Empowered Security Management and Quality of Service Provision in SDN-NFV Environment
39. Ozone depletion identification in stratosphere through faster region-based convolutional neural network
40. Alanazi, S.A., Alruwaili, M., Ahmad, F., Alaerjan, A., and Alshammari, N. (2021). Estimation of Organizational Competitiveness by a Hybrid of One-Dimensional Convolutional Neural Networks and Self-Organizing Maps Using Physiological Signals for Emotional Analysis of Employees. Sensors, 21.10.3390/s21113760 ↗
41. Ciutacu, L. (2022, September 20). Psych-Verbs. Available online: https://github.com/lorenanda/psych-verbs.
42. Yerramsetti, R. (2021, September 20). Speech Emotion Recognition Using SVM Decision Tree and LDA. Available online: https://github.com/RahulYerramsetti/Speech-Emotion-Recognition-Using-SVM-Decision-Tree-and-LDA.
43. Berksudan (2021, September 20). Real Time Emotion Detection. Available online: https://github.com/berksudan/Real-time-Emotion-Detection.
44. Bennett, C., Weiss, B., Suh, J., Yoon, E., Jeong, J., and Chae, Y. (2022). Exploring Data-Driven Components of Socially Intelligent AI through Cooperative Game Paradigms. Multimodal Technol. Interact., 6.10.3390/mti6020016 ↗
45. Kunold, L., and Onnasch, L. (2022). A Framework to Study and Design Communication with Social Robots. Robotics, 11.10.3390/robotics11060129 ↗
46. Human-versus artificial intelligence10.3389/frai.2021.622364 ↗
47. Pandeya, Y.R., Bhattarai, B., and Lee, J. (2021). Deep-learning-based multimodal emotion classification for music videos. Sensors, 21.10.3390/s21144927 ↗
48. Meftah, I.T., Le Thanh, N., and Amar, C.B. (2012, January 3–6). Emotion recognition using KNN classification for user modeling and sharing of affect states. Proceedings of the International Conference on Neural Information Processing, Red Hook, NY, USA.10.1007/978-3-642-34475-6_29 ↗
49. Kovalchuk, Y., Budini, E., Cook, R.M., and Walsh, A. (2022). Investigating the relationship between facial mimicry and empathy. Behav. Sci., 12.10.20944/preprints202206.0267.v1 ↗
50. Naman, A., and Mancini, L. (2021). Fixed-MAML for Few Shot Classification in Multilingual Speech Emotion Recognition. arXiv.10.1007/978-981-16-9650-3_37 ↗
51. Implementation and comparison of speech emotion recognition system using gaussian mixture model (gmm) and k-nearest neighbor (k-nn) techniques10.1016/j.procs.2015.04.226 ↗
52. Navarro-Alamán, J., Lacuesta, R., García-Magariño, I., and Lloret, J. (2022). EmotIoT: An IoT System to Improve Users’ Wellbeing. Appl. Sci., 12.10.3390/app12125804 ↗
53. Muneer, A., and Fati, S.M. (2020). A comparative analysis of machine learning techniques for cyberbullying detection on Twitter. Future Internet, 12.10.3390/fi12110187 ↗
54. Prediction of COVID-19 cases using machine learning for effective public health management
55. Machine learning enabled early detection of breast cancer by structural analysis of mammograms