A review of emotional design in extended reality for the preservation of culture heritage
Source abstract
Abstract
Abstract In recent years, extended reality (XR) technology has revolutionized the cultural heritage sector by enhancing visual and spatial experiences, while evoking emotional responses that deepen users’ engagement with historical narratives. However, research on leveraging emotional interactions in XR to improve the preservation and transmission of intangible cultural heritage remains limited. By conducting a comprehensive literature review, researchers can systematically collect, analyse, and organize existing research, helping to identify key issues, gaps, or trends in this complex field. This review analysed 60 relevant studies to investigate the current applications of XR in the cultural heritage sector. A thematic analysis was conducted to examine how emotional interactions are facilitated through immersive XR experiences. Additionally, potential design methods to improve emotional interaction were explored. The findings reveal that, compared to augmented reality (AR), virtual reality (VR), and mixed reality (MR) has received less attention. Current technologies primarily focus on presenting tangible cultural heritage, including historical buildings and museum collections. Regarding emotional stimulation, most research utilizes interface-driven stimulation, such as visual design elements, highlighting an imbalance in the study of emotional interaction. There is limited exploration of long-term emotional engagement with users. Finally, this review suggests that incorporating gamification and emotional design into XR can significantly benefit the preservation of intangible cultural heritage. This study enhances our understanding of how emotional interaction can facilitate and sustain the transmission of intangible cultural heritage.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Virtual Reality Applications and Impacts · Digital Media and Visual Art · Augmented Reality Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (100+)
Show source-provided references
1. Tribhuvan, A. P. Digital Heritage of Asian Buddhist Caves Using Cultural Computing and AI: Special Reference to Ajanta Caves (2024). In Futuristic trends in artificial intelligence. Vol. 3, 28–37 (International Institute of Projects, 2024).10.58532/V3BKAI11P1CH3 ↗
2. Khalid, A. A. Conservation challenges and emerging trends of digital preservation for UNESCO architectural heritage, Pakistan. Conservation 2, 26–37 (2021).10.3390/conservation2010003 ↗
3. Isa, W. M. W., Zin, N. A. M., Rosdi, F. & Sarim, H. M. Digital preservation of intangible cultural heritage. Indones. J. Electr. Eng. Comput. Sci. 12, 1373 (2018).
4. Economou, M. Heritage in the digital age. In A Companion to Heritage Studies (eds. Logan, W., Craith, M. N. & Kockel, U.) 215–228 (John Wiley & Sons, Inc., 2015).10.1002/9781118486634.ch15 ↗
5. Innocente, C. et al. A metaverse platform for preserving and promoting intangible cultural heritage. Appl. Sci. 14, 3426 (2024).10.3390/app14083426 ↗
6. Sylaiou, S., Dafiotis, P., Fidas, C., Vlachou, E. & Nomikou, V. Evaluating the impact of XR on user experience in the Tomato Industrial Museum “D. Nomikos”. Heritage 7, 1754–1768 (2024).10.3390/heritage7030082 ↗
7. Bruni, L. E., Kadastik, N., Pedersen, T. A. & Dini, H. Digital narratives in extended realities. In Roadmapping Extended Reality, (eds. Alcañiz, M., Sacco, M. & Tromp, J. G.) 35–62 (Scrivener Publishing, 2022).10.1002/9781119865810.ch2 ↗
8. Lee, H. G., Chung, S. & Lee, W. H. Presence in virtual golf simulators: the effects of presence on perceived enjoyment, perceived value, and behavioral intention. N. Media Soc. 15, 930–946 (2013).10.1177/1461444812464033 ↗
9. Lee, Y. C. N., Shan, L. T. & Chen, C. H. System development of immersive technology theatre in museum. In Virtual, Augmented and Mixed Reality. Systems and Applications: 5th International Conference, VAMR 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA, July 21–26, 2013, Proceedings, Part II 5 400–408 (Springer, Berlin, Heidelberg, 2013).10.1007/978-3-642-39420-1_42 ↗
10. Mütterlein, J. The three pillars of virtual reality? Investigating the roles of immersion, presence, and interactivity. In Proc. Hawaii International Conference on System Sciences (ed. Bui, T.) (AIS eLibrary, 2018).10.24251/HICSS.2018.174 ↗
11. Milgram, P. & Kishino, F. A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. E77-D, 1321–1329 (1994).
12. Slater, M. & Wilbur, S. A framework for immersive virtual environments (FIVE): speculations on the role of presence in virtual environments. Presence: Teleoper. Virtual Environ. 6, 603–616 (1997).10.1162/pres.1997.6.6.603 ↗
13. Steuer, J. Defining virtual reality: dimensions determining telepresence. J. Commun. 42, 73–93 (1992).10.1111/j.1460-2466.1992.tb00812.x ↗
14. Azuma, R. T. A survey of augmented reality. Presence.: Teleoper. Virtual Environ. 6, 355–385 (1997).10.1162/pres.1997.6.4.355 ↗
15. Billinghurst, M., Clark, A. & Lee, G. A survey of augmented reality. Found. Trends® Hum.–Comput. Interact. 8, 73–272 (2015).10.1561/1100000049 ↗
16. Azuma, R. T. The challenge of making augmented reality work outdoors. In Mixed Reality: Merging Real and Virtual Worlds (eds. Ohta, Y. & Tamura, H.) 379–390 (Springer-Verlag, 1999).10.1007/978-3-642-87512-0_21 ↗
17. Suh, A. & Prophet, J. The state of immersive technology research: a literature analysis. Comput. Hum. Behav. 86, 77–90 (2018).10.1016/j.chb.2018.04.019 ↗
18. Milgram, P., Takemura, H., Utsumi, A. & Kishino, F. Augmented reality: a class of displays on the reality–virtuality continuum. Proc. Telemanip. Telepresence Technol. 2351, 282–292 (1995).10.1117/12.197321 ↗
19. Zhang, X. & Sung, C. Contextualizing mixed reality: a conceptual framework and survey of applications. Hum.-Centric Comput. Inf. Sci. 7, 1–22 (2017).10.1186/s13673-016-0083-0 ↗
20. Pelanis, E. et al. Use of mixed reality for improved spatial understanding of liver anatomy. Minim. Invasive Ther. Allied. Tehnol. 29, 154–160 (2020).10.1080/13645706.2019.1616558 ↗
21. Speicher, M., Hall, B. D., & Nebeling, M. What is mixed reality? In Proc. 2019 CHI Conference on Human Factors in Computing Systems (eds. Brewster, S. A., Fitzpatrick, G., Cox, A. L., Kostakos, V.) 1–15 (Association for Computing Machinery (ACM), New York, NY, 2019).10.1145/3290605.3300767 ↗
22. Regenbrecht, H., Baratoff, G. & Wilke, W. Augmented reality projects in the automotive and aerospace industries. IEEE Comput. Graph. Appl. 24, 48–56 (2004).10.1109/MCG.2004.1274061 ↗
23. Nocca, F. The role of cultural heritage in sustainable development: multidimensional indicators as decision-making tool. Sustainability 9, 1882 (2017).10.3390/su9101882 ↗
24. Chrysanthakopoulou, A., Kalatzis, K. & Moustakas, K. Immersive virtual reality experience of historical events using haptics and locomotion simulation. Appl. Sci. 11, 11613 (2021).10.3390/app112411613 ↗
25. De Paolis, L. T., Gatto, C., Corchia, L. & De Luca, V. Usability, user experience and mental workload in a mobile augmented reality application for digital storytelling in cultural heritage. Virtual Real. 27, 1117–1143 (2023).10.1007/s10055-022-00712-9 ↗
26. Idris, M. Z., Mustaffa, N. & Yusoff, S. O. S. Preservation of intangible cultural heritage using advance digital technology: issues and challenges. Harmonia: J. Arts Res. Educ. 16, 1 (2016).10.15294/harmonia.v16i1.6353 ↗
27. Doulamis, A., Voulodimos, A., Doulamis, N., Soile, S. & Lampropoulos, A. Transforming intangible folkloric performing arts into tangible choreographic digital objects: the terpsichore approach. In Proc. 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application (eds. Imai, F., Tremeau, A., & Braz, J.) (Scitepress, 2017).10.5220/0006347304510460 ↗
28. Jiaxi, S. Intangible cultural heritage management and protection based on spatial information technology under the background of Internet of Things. Comput. Intell. Neurosci. 2022, 10 (2022).
29. Kong, A. & Wang, X. Chinese Shadow Puppeteering Virtualization Based On a Real-time Finger Tracking System. https://doi.org/10.21203/rs.3.rs-2150731/v1 (2022).10.21203/rs.3.rs-2150731/v1 ↗
30. Yang, C., Lou, M., Chen, X. & Ren, Z. An interactive collaborative creation system for shadow puppets based on smooth generative adversarial networks. Comput. Mater. Contin. 79, 4107–4126 (2024).
31. Shi, Y., Ying, F., Chen, X., Pan, Z. & Yu, J. Restoration of traditional Chinese shadow play‐Piying art from tangible interaction. Comput. Animat. Virtual Worlds 25, 33–43 (2014).10.1002/cav.1530 ↗
32. Ying, X. Inheritance and innovation of Chinese shadow puppetry art in digital media art teaching in colleges and universities. Int. J. N. Dev. Educ. 4, https://doi.org/10.25236/IJNDE.2022.040812 (2022).10.25236/IJNDE.2022.040812 ↗
33. Liang, H., Dong, X., Jy, P. & Zheng, X. Virtual scene generation promotes shadow puppet art conservation. Comput. Animat. Virtual Worlds 34, e2148 (2023).10.1002/cav.2148 ↗
34. Rollins, A. K. Chinese shadow puppetry’s changing apprentice system: questions of continuance and a survey of remaining shadow puppet practitioners in mainland China, 2008–2013. Asian Theatr. J. 32, 295–318 (2015).10.1353/atj.2015.0003 ↗
35. Wei, L. A study on the cultural form and survival ecology of Chinese shadow puppetry. Intang. Cult. Herit. China 01, 14–19 (2022).
36. Gray, J. E. & Bay, M. Platform power, Xr, and the Metaverse: new challenges or old structures? AoIR Sel. Pap. Internet Res. (2023).10.5210/spir.v2023i0.13424 ↗
37. Bowman, D. A. & McMahan, R. P. Virtual reality: how much immersion is enough? Computer 40, 36–43 (2007).10.1109/MC.2007.257 ↗
38. Riva, G. et al. Affective interactions using virtual reality: the link between presence and emotions. Cyberpsychol. Behav. 10, 45–56 (2007).10.1089/cpb.2006.9993 ↗
39. Kurniawan, S. Interaction design: beyond human–computer interaction by Preece, Sharp and Rogers (2001), ISBN 0471492787. Univers. Access Inf. Soc. 3, 289–289 (2004).10.1007/s10209-004-0102-1 ↗
40. Min, X., Sun, S. & Qi, Y. Xr cube: multi-sensory intelligent interaction device for extended reality application. IEEE Access 12, 78058–78073 (2024).10.1109/ACCESS.2024.3401176 ↗
41. Margrett, J. A., Ouverson, K. M., Gilbert, S. B., Phillips, L. A., & Charness, N.. Older adults’ use of extended reality: a systematic review. Front. Virtual Real. 2, 760064 (2022).10.3389/frvir.2021.760064 ↗
42. Bertrand, P., Guegan, J., Robieux, L., McCall, C. A. & Zenasni, F. Learning empathy through virtual reality: multiple strategies for training empathy-related abilities using body ownership illusions in embodied virtual reality. Front. Robot. AI 5, 326671 (2018).10.3389/frobt.2018.00026 ↗
43. Smith, H. J. & Neff, M. Communication behavior in embodied virtual reality. In Proc. 2018 CHI Conference on Human Factors in Computing Systems. 1–12 (Association for Computing Machinery (ACM), New York, NY, 2018).10.1145/3173574.3173863 ↗
44. Santoso, H. B., Wang, J. & Windasari, N. A. Impact of multisensory extended reality on tourism experience journey. J. Hosp. Tour. Technol. 13, 356–385 (2022).
45. Chen, Y. S. & Lin, C. Y. Virtual object replacement based on real environments: potential application in augmented reality systems. Appl. Sci. 9, 1797 (2019).10.3390/app9091797 ↗
46. Chattha, U. A. et al. Motion sickness in virtual reality: an empirical evaluation. IEEE Access 8, 130486–130499 (2020).10.1109/ACCESS.2020.3007076 ↗
47. Panić, N., Leoncini, E., Belvis, G. D., Ricciardi, W. & Boccia, S. Evaluation of the endorsement of the preferred reporting items for systematic reviews and meta-analysis (prisma) statement on the quality of published systematic review and meta-analyses. PLoS ONE 8, e83138 (2013).10.1371/journal.pone.0083138 ↗
48. Rethlefsen, M. L. & Page, M. J. PRISMA 2020 and PRISMA-S: common questions on tracking records and the flow diagram. J. Med. Libr. Assoc. 110, 253–257 (2021).10.5195/jmla.2022.1449 ↗
49. Page, M. J. et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. J. Clin. Epidemiol. 134, 178–189 (2021).10.1016/j.jclinepi.2021.03.001 ↗
50. Braun, V. & Clarke, V. Thematic analysis. Am. Psychol. Assoc. 2, 57–71 (2012).
51. Shi, Z. et al. Evaluating cultural landscape remediation design based on VR technology. ISPRS Int. J. Geo-inf. 10, 423 (2021).10.3390/ijgi10060423 ↗
52. ICH UNESCO. Text of the Convention for the Safeguarding of the Intangible Cultural Heritage 2020 (ICH UNESCO, accessed 21 April 2020); https://ich.unesco.org/en/convention (2003).
53. Bird, J. M. et al. A magic leap in tourism: intended and realized experience of head-mounted augmented reality in a museum context. J. Travel Res. 62, 1427–1447 (2023).10.1177/00472875221134031 ↗
54. Tsai, L. L. Factors that influence virtual tourism holistic image: the moderating role of sense of presence. Sustainability 14, 467 (2022).10.3390/su14010467 ↗
55. Jin, Y., Ma, M. & Zhu, Y. A comparison of natural user interface and graphical user interface for narrative in HMD-based augmented reality. Multimed. Tools Appl. 81, 5795–5826 (2022).10.1007/s11042-021-11723-0 ↗
56. Li, J. Research on display design of intangible cultural heritage based on literary narrative theory: a brief review. J. Educ. Humanit. Soc. Sci. 5, 370–377 (2022).
57. Rizvic, S. et al. Actors in VR storytelling. In 2019 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games) 1–8 (IEEE eXpress Conference Publishing, 2019).10.1109/VS-Games.2019.8864520 ↗
58. Ramy, H., Minhua, M., Ziad, A. & Carl, S. A framework for constructing and evaluating the role of MR as a holographic virtual guide in museums. Virtual Real 25, 895–918 (2021).10.1007/s10055-020-00497-9 ↗
59. Vermesan, O. et al. Edge IoT Industrial Immersive Technologies and Spatial Computing Continuum: Release 1 (Alliance for IoT and Edge Computing Innovation (AIOTI), 2024) https://aioti.eu/wp-content/uploads/AIOTI-Paper-Edge-AI-IoT-Immersive-Technologies-Published.pdf.
60. Guo, X. M., Wang, D. C., Lo, T. T. & Huang, X. H. Historical architecture pedagogy meets virtual technologies: a comparative case study. Educ. Inf. Technol. 29, 14835–14874 (2024).
61. IDC. Worldwide Spending on Augmented and Virtual Reality Forecast to Deliver Strong Growth through 2024, According to a New IDC Spending Guide (IDC, 2023).
62. Vaismoradi, M., Turunen, H. & Bondas, T. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nurs. Health Sci. 15, 398–405 (2013).10.1111/nhs.12048 ↗
63. Braun, V. & Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 3, 77–101 (2006).10.1191/1478088706qp063oa ↗
64. Norman, D. Emotional Design: Why We Love (or Hate) Everyday Things (Basic Books, 2007).
65. Siu, K. W. M. Culture and design: a new burial concept in a densely populated metropolitan area. Des. Issues 21, 79–89 (2005).10.1162/0747936053630151 ↗
66. Norman, D. A. Emotional design. Ubiquity 2004, 1–1 (2004).10.1145/985600.966013 ↗
67. Mu, R. et al. Pilgrimage to Pureland: art, perception and the Wutai Mural VR reconstruction. Int. J. Hum.–Comput. Interact. 40, 2002–2018 (2024).10.1080/10447318.2023.2271665 ↗
68. Taipina, D. & Cardoso, J. C. Spectare: re-designing a stereoscope for a cultural heritage XR experience. Electronics 11, 620 (2022).10.3390/electronics11040620 ↗
69. Jokinen, J. P., Silvennoinen, J., Perälä, P. M. & Saariluoma, P. Quick affective judgments: Validation of a method for primed product comparisons. In Proc. 33rd Annual ACM Conference on Human Factors in Computing Systems (eds. Begole, B., Kim, J., Inkpen, K. & Woo, W.) 2221–2230 (Association for Computing Machinery (ACM), New York, NY, 2015).10.1145/2702123.2702422 ↗
70. Hibbeln, M., Jenkins, J. L., Schneider, C., Valacich, J. S. & Weinmann, M. How is your user feeling? Inferring emotion through human-computer interaction devices. MIS Q. 41, 1–21 (2017).10.25300/MISQ/2017/41.1.01 ↗
71. Lazarus, R. S. Progress on a cognitive-motivational-relational theory of emotion. Am. Psychol. 46, 819–834 (1991).10.1037/0003-066X.46.8.819 ↗
72. Saariluoma, P. & Jokinen, J. P. Emotional dimensions of user experience: a user psychological analysis. Int. J. Hum.-Comput. Interact. 30, 303–320 (2014).10.1080/10447318.2013.858460 ↗
73. Helander, M. & Khalid, H. M. Affective and pleasurable design. In Handbook of Human Factors and Ergonomics (ed. Salvendy, G.) 543–572 (Wiley, 2006).10.1002/0470048204.ch21 ↗
74. Marín‐Morales, J., Llinares, C., Guixeres, J. & Alcañíz, M. Emotion recognition in immersive virtual reality: from statistics to affective computing. Sensors 20, 5163 (2020).10.3390/s20185163 ↗
75. Smith-Harvey, J. & Aguayo, C. Somaesthetics and the non-digital in mixed reality XR education design. Pac. J. Technol. Enhanc. Learn. 4, 38–40 (2022).10.24135/pjtel.v4i1.125 ↗
76. Lazor, M., Gajić, D., Драган, Д & Duță, A. Automation of the avatar animation process in FBX file format. FME Trans. 47, 398–403 (2019).10.5937/fmet1902398L ↗
77. Aristov, M. M., Han, G., Pavelic, A. & Berry, J. F. A new library of 3d models and problems for teaching crystallographic symmetry generated through blender for use with 3d printers or sketchfab. J. Appl. Crystallogr. 55, 172–179 (2022).10.1107/S1600576721013236 ↗
78. Indraprastha, A. & Shinozaki, M. The investigation on using unity3d game engine in urban design study. ITB J. Inf. Commun. Technol. 3, 1–18 (2009).
79. Cowan, B. & Kapralos, B. An overview of serious game engines and frameworks. In Recent Advances in Technologies for Inclusive Well-Being: From Worn to Off-Body Sensing, Virtual Worlds, and Games for Serious Applications (eds. Brooks, A. L., Brahnam, S., Kapralos, B. & Jain, L. C.) 15–38 (Springer, Cham, Switzerland, 2017).10.1007/978-3-319-49879-9_2 ↗
80. Alnagrat, A. J. A., Ismail, R. C. & Idrus, S. Z. S. The opportunities and challenges in virtual reality for virtual laboratories. Innov. Teach. Learn. J. 6, 73–89 (2023).10.11113/itlj.v6.91 ↗
81. Cavalcanti, J. F., Valls, V., Contero, M. & Fonseca, D. Gamification and hazard communication in virtual reality: a qualitative study. Sensors 21, 4663 (2021).10.3390/s21144663 ↗
82. González, E. R., Casals, J. R. & Pérez, S. C. Application of real-time rendering technology to archaeological heritage virtual reconstruction: the example of Casas del Turuñuelo (Guareña, Badajoz, Spain). Virtual Archaeol. Rev. 14, 38–53 (2022).10.4995/var.2023.17460 ↗
83. Lu, L. et al. Applications of mixed reality technology in orthopedics surgery: a pilot study. Front. Bioeng. Biotechnol. 10, 740507 (2022).10.3389/fbioe.2022.740507 ↗
84. Wong, K. C., Sun, Y. E. & Kumta, S. M.. Review and future/potential application of mixed reality technology in orthopaedic oncology. Orthop. Res. Rev. 2022,169–186 (2022).10.2147/ORR.S360933 ↗
85. Rokhsaritalemi, S., Sadeghi-Niaraki, A. & Choi, S. M. A review on mixed reality: current trends, challenges and prospects. Appl. Sci. 10, 636 (2020).10.3390/app10020636 ↗
86. Sun, B. et al. Video conference system in mixed reality using a hololens. Comput. Model. Eng. Sci. 134, 383–403 (2023).
87. Von der Pütten, A. M. et al. Subjective and behavioral presence measurement and interactivity in the collaborative augmented reality game TimeWarp. Interact. Comput. 24, 317–325 (2012).10.1016/j.intcom.2012.03.004 ↗
88. Barrile, V., Bernardo, E., Fotia, A. & Bilotta, G. A combined study of cultural heritage in archaeological museums: 3D survey and mixed reality. Heritage 5, 1330–1349 (2022).10.3390/heritage5030069 ↗
89. Crolla, K. & Goepel, G. Entering hyper-reality: “Resonance-In-Sight,” a mixed-reality art installation. Front. Virtual Real. 3, 1044021 (2022).10.3389/frvir.2022.1044021 ↗
90. Nikolakopoulou, V. et al. Conveying intangible cultural heritage in museums with interactive storytelling and projection mapping: the case of the mastic villages. Heritage 5, 1024–1049 (2022).10.3390/heritage5020056 ↗
91. De Luca, V., Barba, M. C., D’Errico, G., Nuzzo, B. L. & De Paolis, L. T. A user experience analysis for a mobile Mixed Reality application for cultural heritage. Virtual Real. 27, 2821–2837 (2023).10.1007/s10055-023-00840-w ↗
92. Ferretti, U., Quattrini, R. & D’Alessio, M. A comprehensive HBIM to XR framework for museum management and user experience in Ducal palace at Urbino. Heritage 5, 1551–1571 (2022).10.3390/heritage5030081 ↗
93. Olaz, X. et al. An interdisciplinary design of an interactive cultural heritage visit for in-situ, mixed reality and affective experiences. Multimodal Technol. Interact. 6, 59 (2022).10.3390/mti6070059 ↗
94. Szita, K. & Lo, C. H. Visual storytelling and narrative experiences in extended reality: Guest Editors’ introduction. PRESENCE: Virtual Augment. Real. 30, 1–4 (2021).10.1162/pres_e_00384 ↗
95. Li, W. & Cheng, S. Research on user experience model based on mixed reality interactive games. Int. J. Comput. Inf. Technol. 2, 1–1 (2023).
96. Paiva, D. et al. Digital storytelling and hopeful last chance tourism experiences. Tour. Geogr. 25, 1428–1444 (2023).10.1080/14616688.2023.2224043 ↗
97. Agustini, K., Wahyuni, D. S., Mertayasa, I. N. E., Ratminingsih, N. M. & Ariadi, G. The effect of augmented reality mobile application on visitor impact mediated by rational hedonism: evidence from Subak Museum. Int. J. Adv. Comput. Sci. Appl. 14, 77–88 (2023).
98. Ye, J. et al. A study of the effects of historical block context on soundscape perception. Buildings 14, 621 (2024).10.3390/buildings14030621 ↗
99. Assländer, L. & Streuber, S. Virtual reality as a tool for balance research: eyes open body sway is reproduced in photo-realistic, but not in abstract virtual scenes. PLoS ONE 15, e0241479 (2020).10.1371/journal.pone.0241479 ↗
100. Tsita, C. et al. A virtual reality museum to reinforce the interpretation of contemporary art and increase the educational value of user experience. Heritage 6, 4134–4172 (2023).10.3390/heritage6050218 ↗