journal article·2024-09-11Open access

Exploring the Effects of Multi-Factors on User Emotions in Scenarios of Interaction Errors in Human–Robot Interaction

Wa Gao, Yuan Tian, Shiyi Shen, Yang Ji, Ning Sun, Wei Song, Wanli Zhai

Applied Sciences· Volume 14· Issue 18

Abstract

Interaction errors are hard to avoid in the process of human–robot interaction (HRI). User emotions toward interaction errors could further affect the user’s attitudes to robots and experiences of HRI and so on. In this regard, the present study explores the effects of different factors on user emotions when interaction errors occur in HRI. There is sparse research directly studying this perspective. In so doing, three factors, including robot feedback, passive and active contexts, and previous user emotions, were considered. Two stages of online surveys with 465 participants were implemented to explore attitudes to robots and the self-reporting of emotions in active and passive HRI. Then, a Yanshee robot was selected as the experimental platform, and 61 participants were recruited for a real human–robot empirical study based on the two surveys. According to the results of statistical analysis, we conclude some design guides can cope with scenarios of interaction errors. For example, feedback and previous emotions have impacts on user emotions after encountering interaction errors, but contexts do not. There are no interactive effects between the three factors. The approach to reduce negative emotions in the cases of interaction errors in HRI, such as providing irrelevant feedback and so on, is also illustrated in the contributions.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Social Robot Interaction and HRI · Emotion and Mood Recognition · Color perception and design

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Co-Innovation Center of Efficient Processing and Utilization of Forest Resources, Nanjing Forestry University, Nanjing 210037, China

College of Furnishings and Industrial Design, Nanjing Forestry University, Nanjing 210037, China

College of Automobile and Traffic Engineering, Nanjing Forestry University, Nanjing 210037, China

Funding

National Natural Science Foundation of China

References (51)

Show source-provided references
1. Human-like Guide Robot that Proactively Explains Exhibits10.1007/s12369-019-00587-y ↗
2. Estimating Children’s Social Status through Their Interaction Activities in Classrooms with a Social Robot10.1007/s12369-018-0474-7 ↗
3. Stop Ignoring Me! On Fighting the Trivialization of Social Robots in Public Spaces10.1145/3488241 ↗
4. Controlling Robots in the Home: Factors that Affect the Performance of Novice Robot Operators10.1016/j.apergo.2017.05.005 ↗
5. Mirnig, N., Giuliani, M., Stollnberger, G., Stadler, S., Buchner, R., and Tscheligi, M. (2015, January 26–30). Impact of Robot Actions on Social Signals and Reaction Times in HRI Error Situations. Proceedings of the International Conference on Social Robotics, Paris, France.10.1007/978-3-319-25554-5_46 ↗
6. A Taxonomy of Social Errors in Human-Robot Interaction10.1145/3439720 ↗
7. Humor by Violating an Existing Expectancy10.1080/00223980.1981.9915251 ↗
8. Lopez, S.J., and Snyder, C.R. (2009). A Role for Neuropsychology in Understanding the Facilitating Influence of Positive Affect on Social Behavior and Cognitive Processes. Oxford Handbook of Positive Psychology, Oxford University Press. [1st ed.].10.1093/oxfordhb/9780195187243.013.0048 ↗
9. Review of How Emotions Are Made: The Secret Life of the Brain
10. The Future of Emotion Regulation Research: Capturing Context10.1177/1745691612459518 ↗
11. Li, W., Shao, W., Ji, S., and Cambria, E. (2020). BiERU: Bidirectional Emotional Recurrent Unit for Conversational Sentiment Analysis. arXiv.
12. Robot Learning from Human Demonstrations with Inconsistent Contexts10.1016/j.robot.2023.104466 ↗
13. Context-Enhanced Human-Robot Interaction: Exploring the Role of System Interactivity and Multimodal Stimuli on the Engagement of People with Dementia10.1007/s12369-021-00823-4 ↗
14. Kim, S.K., Kirchner, E.A., Schloßmüller, L., and Kirchner, F. (2020). Errors in Human-Robot Interactions and Their Effects on Robot Learning. Front. Robot. AI, 7.10.3389/frobt.2020.558531 ↗
15. van der, Z.T., Chen, X., Stone, P.L., and Sucar, E. (2013). A Survey about Faults of Robots Used in RoboCup, Lecture Notes in Computer Science, Springer.
16. Kwon, M., Huang, S.H., and Dragan, A.D. (2018, January 23). Expressing Robot Incapability. Proceedings of the Thirteenth Annual ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA.10.1145/3171221.3171276 ↗
17. Mirnig, N., Stollnberger, G., Miksch, M., Stadler, S., Giuliani, M., and Tscheligi, M. (2017). To Err Is Robot: How Humans Assess and Act Toward an Erroneous Social Robot. Front. Robot. AI, 4.10.3389/frobt.2017.00021 ↗
18. Lucas, G.M., Boberg, J., Traum, D., Artstein, R., Gratch, J., Gainer, A., Johnson, E., Leuski, A., and Nakano, M. (2018, January 26). Getting to Know Each Other: The Role of Social Dialogue in Recovery from Errors in Social Robots. Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA.10.1145/3171221.3171258 ↗
19. Lucas, G.M., Boberg, J., Traum, D., Artstein, R., Gratch, J., Gainer, A., Johnson, E., Leuski, A., and Nakano, M. (2017, January 17). The Role of Social Dialogue and Errors in Robots. Proceedings of the 5th International Conference on Human Agent Interaction, New York, NY, USA.10.1145/3125739.3132617 ↗
20. Honig, S., and Oron-Gilad, T. (2018). Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development. Front. Psychol., 9.10.3389/fpsyg.2018.00861 ↗
21. Giuliani, M., Mirnig, N., Stollnberger, G., Stadler, S., Buchner, R., and Tscheligi, M. (2015). Systematic Analysis of Video Data from Different Human–Robot Interaction Studies: Acategorization of Social Signals During Error Situations. Front. Psychol., 6.10.3389/fpsyg.2015.00931 ↗
22. Chen, W., Zhou, C., Shang, G., Wang, X., Li, Z., Xu, C., and Hu, K. (2022). SLAM Overview: From Single Sensor to Heterogeneous Fusion. Remote Sens., 14.10.3390/rs14236033 ↗
23. Fahn, C.-S., Chen, S.-C., Wu, P.-Y., Chu, T.-L., Li, C.-H., Hsu, D.-Q., Wang, H.-H., and Tsai, H.-M. (2022). Image and Speech Recognition Technology in the Development of an Elderly Care Robot: Practical Issues Review and Improvement Strategies. Healthcare, 10.10.3390/healthcare10112252 ↗
24. NAO Robot Teleoperation with Human Motion Recognition10.1007/s13369-021-06051-2 ↗
25. Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., and Wong, W.K. (2013, January 23–26). Too Much, Too Little, or Just Right? Ways Explanations Impact End Users’ Mental Models. Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing, San Jose, CA, USA.10.1109/VLHCC.2013.6645235 ↗
26. Robot Apology as a Post-Accident Trust-Recovery Control Strategy in Industrial Human-Robot Interaction10.1016/j.ergon.2020.103078 ↗
27. Interaction Strategy for Robotic Apology Based on Human Orientation Toward Service10.1080/01691864.2024.2321173 ↗
28. When Your Robot Avatar Misbehaves You Are Likely to Apologize: An Exploration of Guilt During Robot Embodiment10.1007/s12369-019-00556-5 ↗
29. Analyzing the Impact of Human Errors on Interactive Service Robotic Scenarios Via Formal Verification10.1007/s10270-023-01125-1 ↗
30. Small-Sized Handshake Robot System for Generation of Handshake Behavior with Active Approach to Human10.1299/jamdsm.2019jamdsm0026 ↗
31. Okuda, M., Takahashi, Y., and Tsuichihara, S. (2022). Human Response to Humanoid Robot That Responds to Social Touch. Appl. Sci., 12.10.3390/app12189193 ↗
32. Survey of Emotions in Human–Robot Interactions: Perspectives from Robotic Psychology on 20 Years of Research10.1007/s12369-021-00778-6 ↗
33. Risk Choice and Emotional Experience: A Multi-Level Comparison between Active and Passive Decision-making10.1080/13669877.2018.1459798 ↗
34. Toward Active Physical Human–Robot Interaction: Quantifying the Human State During Interactions10.1109/THMS.2021.3138684 ↗
35. Interact with Me: An Exploratory Study on Interaction Factors for Active Physical Human-Robot Interaction10.1109/LRA.2020.3017475 ↗
36. Zhang, B., Nakamura, T., Kaneko, M., and Lim, H.O. (2020, January 12–15). Development of an Autonomous Guide Robot based on Active Interactions with Users. Proceedings of the 2020 IEEE/SICE International Symposium on System Integration, Honolulu, HI, USA.10.1109/SII46433.2020.9026301 ↗
37. Robocop: A Robotic Coach for Oral Presentations10.1145/3090092 ↗
38. The Future of Service: The Power of Emotion in Human-Robot Interaction10.1016/j.jretconser.2021.102551 ↗
39. Tourists’ perceptions regarding the use of anthropomorphic robots in tourism and hospitality10.1108/IJCHM-05-2020-0423 ↗
40. Emotion as an Emergent Phenomenon of the Neurocomputational Energy Regulation Mechanism of a Cognitive Agent in a Decision-Making Task10.1177/1059712319880649 ↗
41. Gratch, J. (2000, January 1). Émile: Marshalling Passions in Training and Education. Proceedings of the Fourth International Conference on Autonomous Agents, Brasília, Brazil.10.1145/336595.337516 ↗
42. Kirtay, M., Vannucci, L., Falotico, E., Oztop, E., and Laschi, C. (2016, January 12). Sequential Decision Making Based on Emergent Emotion for a Humanoid Robot. Proceedings of the IEEE-RAS International Conference on Humanoid Robots (Humanoids 2016), Tokyo, Japan.10.1109/HUMANOIDS.2016.7803408 ↗
43. Kirtay, M., and Oztop, E. (2013, January 12–15). Emergent Emotion Via Neural Computational Energy Conservation on a Humanoid Robot. Proceedings of the 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids), Tokyo, Japan.10.1109/HUMANOIDS.2013.7030013 ↗
44. A Motivational Model Based on Artificial Biological Functions for the Intelligent Decision-making of Social Robots10.1007/s12293-023-00390-3 ↗
45. Ahn, H., and Park, S. (2021). Contextual Emotion Appraisal Based on a Sentential Cognitive System for Robots. Appl. Sci., 11.10.20944/preprints202007.0498.v2 ↗
46. Behavior Models of Emotion-Featured Robots: A Survey10.1007/s10846-020-01219-8 ↗
47. Spezialetti, M., Placidi, G., and Rossi, S. (2020). Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives. Front. Robot. AI, 7.10.3389/frobt.2020.532279 ↗
48. Hirth, J., Schmitz, N., and Berns, K. (2007, January 10–14). Emotional Architecture for the Humanoid Robot Head ROMAN. Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Rome, Italy.10.1109/ROBOT.2007.363639 ↗
49. What Kinds of Robot’s Touch Will Match Expressed Emotions?10.1109/LRA.2019.2947010 ↗
50. The Effects of Overall Robot Shape on the Emotions Invoked in Users and the Perceived Personalities of Robot10.1016/j.apergo.2012.10.010 ↗
51. Cirasa, C., Høgsdal, H., and Conti, D. (2024). “I See What You Feel”: An Exploratory Study to Investigate the Understanding of Robot Emotions in Deaf Children. Appl. Sci., 14.10.20944/preprints202401.0816.v1 ↗