Exploring the Impacts of Robot Voice, Appearance, and Ambient Sound on User Experience in Museums
Source abstract
Abstract
Few studies have examined robot voice, appearance, and ambient sound within a single framework to understand the user experience. To address this gap, this study explores how these three factors influence the user experience, specifically focusing on the appearance-voice matching effect, user likability, and user satisfaction when robots serve as museum docents, thereby offering design-oriented strategic insights for robot deployment in museums. Audio recordings of museum docents were collected and clustered into four categories. Accordingly, four distinct voice models were synthesized, and subjective user perceptions of these models were evaluated. Subsequently, three different agents, including an iPad, a self-designed robot prototype and a Yanshee robot, were employed as experimental platforms. These agents were each equipped with different voice models to conduct the experiments. The experimental data were analyzed by repeated measures ANOVA and mixed-model analysis. The results indicate that robot appearance significantly affects the appearance-voice matching effect, while its voice moderates this relationship in a museum scenario. Robot voice primarily drives user likability and satisfaction. In contrast, robot appearance shapes the two dimensions differently. Appropriate ambient sound significantly enhances the perceived appearance-voice congruence, and significantly affects naturalness and liveliness within user likability, as well as the fun of the experience and pleasantness of use within user satisfaction. The congruence can positively predict user likability and satisfaction, but this predictive effect has boundaries. These findings can provide references from the perspective of design strategies for human–robot interaction in museums.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Social Robot Interaction and HRI · AI in Service Interactions · Multisensory perception and integration
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Co-Innovation Center of Efficient Processing and Utilization of Forest Resources, Nanjing Forestry University, Nanjing 210037, China
College of Furnishings and Industrial Design, Nanjing Forestry University, Nanjing 210037, China
Funding
National Natural Science Foundation of China · Jiangsu Provincial Social Science Fund Project
References (66)
Show source-provided references
1. Deploying social robots in museum settings: A quasi-systematic review exploring purpose and acceptability10.1177/17298814211066740 ↗
2. Rosa, S., Randazzo, M., Landini, E., Bernagozzi, S., Sacco, G., Piccinino, M., and Natale, L. (2024). Tour guide robot: A 5G-enabled robot museum guide. Front. Robot. AI, 10.10.3389/frobt.2023.1323675 ↗
3. The use of robots in the workplace: Conclusions from a health promoting intervention using social robots10.1007/s12369-023-01000-5 ↗
4. Towards a more anthropomorphic interaction with robots in museum settings: An experimental study10.1016/j.robot.2023.104561 ↗
5. The mechanism of value cocreation in robotic services: Customer inspiration from robotic service novelty
6. Customer-robot interactions: Understanding customer experience with service robot10.1016/j.ijhm.2021.103078 ↗
7. Sharing experiences to help a robot present its mind and sociability10.1007/s12369-020-00643-y ↗
8. Kamino, W., Abanović, S., and Jung, M.F. (2024, January 26–30). The lifecycle of social robots: Obsolescence and values in repair. Proceedings of the 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Pasadena, CA, USA.10.1109/RO-MAN60168.2024.10731206 ↗
9. Zhong, V.J., Mürset, N., Jäger, J., and Schmiedel, T. (2022, January 7–10). Exploring variables that affect robot likeability. Proceedings of the 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Sapporo, Japan.10.1109/HRI53351.2022.9889602 ↗
10. Schreibelmayr, S., and Mara, M. (2022). Robot voices in daily life: Vocal human-likeness and application context as determinants of user acceptance. Front. Psychol., 13.10.3389/fpsyg.2022.787499 ↗
11. A cross-cultural study: Effect of robot appearance and task10.1007/s12369-010-0056-9 ↗
12. Torre, I., Latupeirissa, A.B., and McGinn, C. (September, January 31). How context shapes the appropriateness of a robot’s voice. Proceedings of the 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Naples, Italy.10.1109/RO-MAN47096.2020.9223449 ↗
13. A mismatch in the human realism of face and voice produces an uncanny valley10.1068/i0415 ↗
14. The uncanny effect of speech: The impact of appearance and speaking on impression formation in human–robot interactions10.1007/s12369-023-00976-4 ↗
15. Salem, M., Ziadee, M., and Sakr, M. (2013, January 27–29). Effects of politeness and interaction context on perception and experience of HRI. Proceedings of the 5th International Conference on Social Robotics (ICSR), Bristol, UK.10.1007/978-3-319-02675-6_53 ↗
16. A music interaction system based on the emotional recognition of older adults in smart homes: Cross-sectional experimental study10.2196/77218 ↗
17. Soundscapes in home environments: The impact of home types and soundscapes on the recovery benefits of psychophysiological stress
18. Need satisfaction in human-robot interactions in public spaces—A positive computing approach10.1007/s12369-025-01336-0 ↗
19. To err is human(-like): Effects of robot gesture on perceived anthropomorphism and likability10.1007/s12369-013-0196-9 ↗
20. Technical methods for social robots in museum settings: An overview of the literature10.1007/s12369-022-00904-y ↗
21. Human-like guide robot that proactively explains exhibits10.1007/s12369-019-00587-y ↗
22. Gehle, R., Pitsch, K., Dankert, T., and Wrede, S. (2017, January 6–9). How to open an interaction between robot and museum visitor? Strategies to establish a focused encounter in HRI. Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (HRI), Vienna, Austria.10.1145/2909824.3020219 ↗
23. Díaz, M., Paillacho, D., Angulo, C., Torres, O., González, J., and Albo-Canals, J. (2014, January 3–6). A week-long study on robot-visitors spatial relationships during guidance in a sciences museum. Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction (HRI), Bielefeld, Germany.10.1145/2559636.2559797 ↗
24. Museum robot guides or conventional audio guides? An experimental study10.1080/01691864.2020.1854113 ↗
25. Yoon, H., Kim, M.-G., Kim, S., and Suh, J.-H. (2025). Designing personalization cues for museum robots: Docent observation and controlled studies. Sensors, 25.10.3390/s25227095 ↗
26. Consumer resistance to service robots at the hotel front desk: A mixed-methods research
27. Keskin, M.O., Akay, S., Dogan, A., Koçyigit, B., Kanero, J., and Aydogan, R. (2024, January 11–15). You look nice, but I am here to negotiate: The influence of robot appearance on negotiation dynamics. Proceedings of the Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI), Boulder, CO, USA.10.1145/3610978.3640759 ↗
28. Avoiding the uncanny valley: Robot appearance, personality, and consistency of behavior in an attention seeking home scenario for a robot companion10.1007/s10514-007-9058-3 ↗
29. The effects of appearance personification of service robots on customer decision-making in the product recommendation context10.1108/IMDS-11-2021-0662 ↗
30. FakhrHosseini, S.M., Hilliger, S., Barnes, J., Jeon, M., Park, C.H., and Howard, A.M. (2017, January 28–31). Love at first sight: Mere exposure to robot appearance leaves impressions similar to interactions with physical robots. Proceedings of the 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Lisbon, Portugal.10.1109/ROMAN.2017.8172366 ↗
31. Robotic faciality: The philosophy, science and art of robot faces10.1007/s12369-020-00623-2 ↗
32. Emotional experiences of service robots’ anthropomorphic appearance: A multimodal measurement method10.1080/00140139.2023.2182751 ↗
33. Unveiling the role of congruity in service robot design and deployment10.1108/IJCHM-11-2023-1686 ↗
34. Voges, A., Foster, M.E., and Cross, E.S. (2024, January 26–30). Human, animal, or machine? A design-based exploration of social robot embodiment with a creative toolkit. Proceedings of the 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Pasadena, CA, USA.10.1109/RO-MAN60168.2024.10731416 ↗
35. Song, Y., Luximon, A., and Luximon, Y. (2023). Facial anthropomorphic trustworthiness scale for social robots: A hybrid approach. Biomimetics, 8.10.3390/biomimetics8040335 ↗
36. Zhang, J., Zhu, X., Tang, W., Ge, W., Zhang, Y., and Li, J. (2026). Enhancing Alarm Localization in Multi-Window Map Interfaces with Spatialized Auditory Cues: An Eye-Tracking Study. ISPRS Int. J. Geo-Inf., 15.10.3390/ijgi15020069 ↗
37. Effects of lateral head tilt on user perceptions of humanoid and android robots10.1016/j.chb.2014.09.025 ↗
38. McGinn, C., and Torre, I. (2019, January 11–14). Can you tell the robot by the voice? An exploratory study on the role of voice in the perception of robots. Proceedings of the 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Daegu, Republic of Korea.10.1109/HRI.2019.8673305 ↗
39. Mara, M., Schreibelmayr, S., and Berger, F. (2020, January 23–26). Hearing a nose? User expectations of robot appearance induced by different robot voices. Proceedings of the Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI), Cambridge, UK.10.1145/3371382.3378285 ↗
40. Kavuza, N.F., Garcia Goo, H., and Truong, K.P. (2025, January 15–18). Do I sound as capable as I look? Impact of robot communication style and appearance on user perception. Proceedings of the 25th ACM International Conference on Intelligent Virtual Agents (IVA), Chicago, IL, USA.
41. Van Rijn, P., Mertes, S., Janowski, K., Weitz, K., Jacoby, N., and André, E. (2024, January 11–16). Giving robots a voice: Human-in-the-loop voice creation and open-ended labeling. Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24), Honolulu, HI, USA.10.1145/3613904.3642038 ↗
42. Audiovisual affective design of humanoid robot appearance and voice based on Kansei engineering10.1007/s12369-024-01202-5 ↗
43. Multimodal person evaluation: First impressions from faces, voices, and names10.1037/pspa0000454 ↗
44. Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots10.1007/s12369-008-0001-3 ↗
45. Gao, W., Cen, G., Zhai, W., Gao, F., Li, J., and Miao, Y. (2024). Exploring the effects of multi-factors on user emotions in scenarios of interaction errors in human–robot interaction. Appl. Sci., 14.10.3390/app14188164 ↗
46. Robot likeability and reciprocity in human robot interaction: Using ultimatum game to determine reciprocal likeable robot strategies
47. Effects of form and motion on judgments of social robots’ animacy, likability, trustworthiness and unpleasantness10.1016/j.ijhcs.2016.02.004 ↗
48. Lehmann, H., Saez-Pons, J., Syrdal, D.S., and Dautenhahn, K. (2015). In good company? Perception of movement synchrony of a non-anthropomorphic robot. PLoS ONE, 10.10.1371/journal.pone.0127747 ↗
49. Investigation of robot expression style in human-robot interaction10.20965/jrm.2020.p0224 ↗
50. Why some humanoid faces are perceived more positively than others: Effects of human-likeness and task10.1007/s12369-014-0269-4 ↗
51. Understanding the effect of anthropomorphic features of humanoid social robots on user satisfaction: A stimulus-organism-response approach10.1108/IMDS-10-2023-0781 ↗
52. Users’ responses to humanoid social robots: A social response view10.1016/j.tele.2024.102146 ↗
53. Shi, Y., Zhan, W., and Lin, C. (2025). The impact of hotel robots’ service quality on continuance intention: The moderating effect of personal innovation. Front. Robot. AI, 12.10.3389/frobt.2025.1667123 ↗
54. Shin, S., Ko, S.B., and Kang, J. (2026). Sustainable tourist satisfaction in art museums: Identifying attributes that enhance visitor experience for sustainable cultural management. Sustainability, 18.10.3390/su18031367 ↗
55. Seo, K.H., and Lee, J.H. (2021). The emergence of service robots at restaurants: Integrating trust, perceived risk, and satisfaction. Sustainability, 13.10.3390/su13084431 ↗
56. Seo, K.H., and Lee, J.H. (2025). The impact of service quality on perceived value, image, satisfaction, and revisit intention in robotic restaurants for sustainability. Sustainability, 17.10.3390/su17167422 ↗
57. Phan, T.L., and Cosgun, A. (2025). Placing objects on table is preferred over direct handovers when users are occupied. Sensors, 25.10.20944/preprints202501.1115.v1 ↗
58. Kwon, J.Y., and Ju, D.Y. (2023). Living lab-based service interaction design for a companion robot for seniors in South Korea. Biomimetics, 8.10.3390/biomimetics8080609 ↗
59. Zhang, A.W., Queiroz, R., and Sebo, S.S. (2025, January 4–6). Balancing user control and perceived robot social agency through the design of end-user robot programming interfaces. Proceedings of the 2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Melbourne, Australia.10.1109/HRI61500.2025.10974063 ↗
60. No more mumbles: Enhancing robot intelligibility through speech adaptation10.1109/LRA.2024.3401117 ↗
61. Physicochemical characteristics and their variation of coal dust originating from underground mining sites10.1021/acsomega.4c06729 ↗
62. Mirnig, N., Stadler, S., Stollnberger, G., Giuliani, M., and Tscheligi, M. (2016, January 26–31). Robot humor: How self-irony and Schadenfreude influence people’s rating of robot likability. Proceedings of the 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), New York, NY, USA.10.1109/ROMAN.2016.7745106 ↗
63. Dantas, C., Jegundo, A.L., Quintas, J., Martins, A.I., Queirós, A., and Rocha, N.P. (2017, January 11–12). European Portuguese validation of usefulness, satisfaction and ease of use questionnaire (USE). Proceedings of the World Conference on Information Systems and Technologies (WorldCIST’17), Madeira, Portugal.10.1007/978-3-319-56538-5_57 ↗
64. A meta-analysis on the effectiveness of anthropomorphism in human-robot interaction10.1126/scirobotics.abj5425 ↗
65. Hsu, L.J., Khoo, W., Randall, N., Kamino, W., Joshi, S., Sato, H., Crandall, D.J., Tsui, K.M., and Šabanović, S. (2023, January 28–31). Finding its Voice: The Influence of Robot Voice on Fit, Social Attributes, and Willingness to Use Among Older Adults in the U.S. and Japan. Proceedings of the 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Busan, Republic of Korea.10.1109/RO-MAN57019.2023.10309390 ↗
66. Age-related differences in the evaluation of a virtual health agent’s appearance and embodiment in a health-related interaction: Experimental lab study10.2196/13726 ↗