X-Reality Museums: Unifying the Virtual and Real World Towards Realistic Virtual Museums
Source abstract
Abstract
Culture is a field that is currently entering a revolutionary phase, no longer being a privilege for the few, but expanding to new audiences who are urged to not only passively consume cultural heritage content, but actually participate and assimilate it on their own. In this context, museums have already embraced new technologies as part of their exhibitions, many of them featuring augmented or virtual reality artifacts. The presented work proposes the synthesis of augmented, virtual and mixed reality technologies to provide unified X-Reality experiences in realistic virtual museums, engaging visitors in an interactive and seamless fusion of physical and virtual worlds that will feature virtual agents exhibiting naturalistic behavior. Visitors will be able to interact with the virtual agents, as they would with real world counterparts. The envisioned approach is expected to not only provide refined experiences for museum visitors, but also achieve high quality entertainment combined with more effective knowledge acquisition.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Augmented Reality Applications · Virtual Reality Applications and Impacts · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Foundation for Research and Technology Hellas, Institute of Computer Science, N. Plastira 100, Vassilika Vouton, GR-700 13 Heraklion, Greece
Department of Computer Science, University of Crete, GR-700 13 Heraklion, Greece
References (78)
Show source-provided references
1. Sacco, P.L. (2011). Culture 3.0: A New Perspective for the EU 2014–2020 Structural Funds Programming, European Expert Network on Culture.
2. Towards a new era for Cultural Heritage Education: Discussing the role of ICT10.1016/j.chb.2010.07.031 ↗
3. Ioannides, M., and Davies, R. (2018, January 25–27). ViMM-Virtual Multimodal Museum: A Manifesto and Roadmap for Europe’s Digital Cultural Heritage. Proceedings of the IEEE 2018 International Conference on Intelligent Systems, Funchal, Portugal.10.1109/IS.2018.8710556 ↗
4. Sylaiou, S., Kasapakis, V., Dzardanova, E., and Gavalas, D. (2018, January 25–27). Leveraging mixed reality technologies to enhance museum visitor experiences. Proceedings of the IEEE 2018 International Conference on Intelligent Systems, Funchal, Portugal.10.1109/IS.2018.8710530 ↗
5. Majd, M., and Safabakhsh, R. (2017, January 25–27). Impact of machine learning on improvement of user experience in museums. Proceedings of the IEEE 2017 Artificial Intelligence and Signal Processing (AISP), Shiraz, Iran.10.1109/AISP.2017.8324080 ↗
6. Discovering Leonardo with artificial intelligence and holograms: A user study10.1016/j.patrec.2020.01.006 ↗
7. Kiourt, C., Pavlidis, G., Koutsoudis, A., and Kalles, D. (2017, January 26–28). Multi-agents based virtual environments for cultural heritage. Proceedings of the 2017 IEEE XXVI International Conference on Information, Communication and Automation Technologies (ICAT), Sarajevo, Bosnia and Herzegovina.10.1109/ICAT.2017.8171602 ↗
8. (2020, December 21). Next Generation Internet—Interactive Technologies. Available online: https://ec.europa.eu/digital-single-market/en/next-generation-internet-interactive-technologies.
9. (2020, December 21). 5 Trends Emerge in the Gartner Hype Cycle for Emerging Technologies. Available online: //www.gartner.com/smarterwithgartner/5-trends-emerge-in-gartner-hype-cycle-for-emerging-technologies-2018/.
10. Ptukhin, A., Serkov, K., Khrushkov, A., and Bozhko, E. (2018, January 7–8). Prospects and modern technologies in the development of VR/AR. Proceedings of the IEEE 2018 Ural Symposium on Biomedical Engineering, Radioelectronics and Information Technology, Yekaterinburg, Russia.10.1109/USBEREIT.2018.8384578 ↗
11. A survey of augmented, virtual, and mixed reality for cultural heritage10.1145/3145534 ↗
12. Cochrane, N. (2020, November 24). VFX-1 Virtual Reality Helmet by Forte. Game Bytes Magazine, 11 November 1994, p. 21. Available online: http://www.ibiblio.org/GameBytes/issue21/flooks/vfx1.html.
13. Burdea, G.C., and Coiffet, P. (2017). Virtual Reality Technology, John Wiley & Sons. [2nd ed.].
14. Why virtual reality will transform a workplace near you10.1145/3105444 ↗
15. Heinonen, M. (2017). Adoption of VR and AR Technologies in the Enterprise. [Master’s Thesis, Lappeenranta University of Technology].
16. PricewaterhouseCoopers (2020, December 21). A Decade of Digital: Keeping Pace with Transformation. Available online: https://www.pwc.com/us/en/advisory-services/digital-iq/assets/pwc-digital-iq-report.pdf.
17. A survey of augmented reality10.1561/1100000049 ↗
18. Henderson, S.J., and Feiner, S.K. (2011, January 26–29). Augmented reality in the psychomotor phase of a procedural task. Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland.10.1109/ISMAR.2011.6162888 ↗
19. A survey of augmented reality technologies, applications and limitations10.20870/IJVR.2010.9.2.2767 ↗
20. Boom, D.V. (2020, December 21). Pokemon Go Has Crossed 1 Billion in Downloads. Available online: https://www.cnet.com/news/pokemon-go-has-crossed-1-billion-in-downloads/.
21. A taxonomy of mixed reality visual displays
22. Ohta, Y., and Tamura, H. (2014). Mixed Reality: Merging Real and Virtual Worlds, Springer.
23. Augmenting natural interaction with physical paper in ambient intelligence environments10.1007/s11042-018-7088-9 ↗
24. Tracking of multiple planar projection boards for interactive mixed-reality applications10.1007/s11042-017-5313-6 ↗
25. Margetis, G., Grammenos, D., Zabulis, X., and Stephanidis, C. (2013, January 21–26). iEat: An interactive table for restaurant customers’ experience enhancement. Proceedings of the International Conference on Human-Computer Interaction 2013, Las Vegas, NV, USA.10.1007/978-3-642-39476-8_134 ↗
26. AR teleport: Digital reconstruction of historical and cultural-heritage sites for mobile phones via movement-based interactions10.1007/s11277-012-0758-4 ↗
27. Papadaki, E., Zabulis, X., Ntoa, S., Margetis, G., Koutlemanis, P., Karamaounas, P., and Stephanidis, C. (2013, January 15–19). The book of Ellie: An interactive book for teaching the alphabet to children. Proceedings of the 2013 IEEE International Conference on Multimedia and Expo Workshops, San Jose, CA, USA.10.1109/ICMEW.2013.6618341 ↗
28. Ioannides, M., Magnenat-Thalmann, N., and Papagiannakis, G. (2017). Gamified AR/VR character rendering and animation-enabling technologies. Mixed Reality and Gamification for Cultural Heritage, Springer.10.1007/978-3-319-49607-8 ↗
29. Karakottas, A., Papachristou, A., Doumanoqlou, A., Zioulis, N., Zarpalas, D., and Daras, P. (2018, January 18–22). Augmented VR. Proceedings of the 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Reutlingen, Germany.10.1109/VR.2018.8446561 ↗
30. Simsarian, K.T., and Akesson, K.P. (1997, January 28–30). Windows on the world: An example of augmented virtuality. Proceedings of the 6th International Conference on Man-Machine Interaction Intelligent Systems in Business, Montpellier, France.
31. Regenbrecht, H., Ott, C., Wagner, M., Lum, T., Kohler, P., Wilke, W., and Mueller, E. (2003, January 10). An augmented virtuality approach to 3D videoconferencing. Proceedings of the Second IEEE and ACM International Symposium on Mixed and Augmented Reality, Tokyo, Japan.
32. Apostolakis, K.C., Alexiadis, D.S., Daras, P., Monaghan, D., O’Connor, N.E., Prestele, B., Eisert, P., Richard, G., Zhang, Q., and Izquierdo, E. (2013, January 3–5). Blending real with virtual in 3DLife. Proceedings of the 14th International IEEE Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2013), Paris, France.10.1109/WIAMIS.2013.6616123 ↗
33. Immersing users in landscapes using large scale displays in public spaces
34. Space Wars: An AugmentedVR Game
35. Grammenos, D., Margetis, G., Koutlemanis, P., and Zabulis, X. (2012, January 3–5). Paximadaki, the game: Creating an advergame for promoting traditional food products. Proceedings of the 16th International Academic MindTrek Conference, Tampere, Finland.10.1145/2393132.2393195 ↗
36. Zikas, P., Bachlitzanakis, V., Papaefthymiou, M., Kateros, S., Georgiou, S., Lydatakis, N., and Papagiannakis, G. (2016, January 6–7). Mixed reality serious games and gamification for smart education. Proceedings of the 2016 European Conference on Games Based Learning, Paisley, UK.
37. Albert, A., Hallowell, M.R., Kleiner, B., Chen, A., and Golparvar-Fard, M. (2014). Enhancing construction hazard recognition with high-fidelity augmented virtuality. J. Constr. Eng. Manag., 140.10.1061/(ASCE)CO.1943-7862.0000860 ↗
38. Chen, A., Golparvar-Fard, M., and Kleiner, B. (2013, January 30–31). SAVES: A safety training augmented virtuality environment for construction hazard recognition and severity identification. Proceedings of the 13th International Conference on Construction Applications of Virtual Reality, London, UK.10.1061/9780784413029.105 ↗
39. Augmented virtuality based on stereoscopic reconstruction in multimodal image-guided neurosurgery: Methods and performance evaluation10.1109/TMI.2005.857029 ↗
40. Using Sensor Inputs to Affect Virtual and Real Environments10.1109/MPRV.2009.60 ↗
41. Mann, S., Havens, J.C., Iorio, J., Yuan, Y., and Furness, T. (June, January 30). All Reality: Values, taxonomy, and continuum, for Virtual, Augmented, eXtended/MiXed (X), Mediated (X, Y), and Multimediated Reality/Intelligence. Proceedings of the AWE 2018 Conference, Santa Clara, CA, USA.
42. Testing and validating Extended Reality (xR) technologies in manufacturing10.1016/j.promfg.2018.06.054 ↗
43. Lee, Y., Moon, C., Ko, H., Lee, S.H., and Yoo, B. (2020, January 9–13). Unified Representation for XR Content and its Rendering Method. Proceedings of the 25th ACM International Conference on 3D Web Technology, Seoul, Korea.10.1145/3424616.3424695 ↗
44. (2020, November 24). Extended Reality (XR = Augmented Reality + Virtual Reality + Mixed Reality) Marketplace 2018–2023. Available online: https://www.researchandmarkets.com/reports/4658849/extended-reality-xr-augmented-reality.
45. Schreibman, S., Siemens, R., and Unsworth, J. (2015). Embodiment, Entanglement, and Immersion in Digital Cultural Heritage. A New Companion to Digital Humanities, John Wiley & Sons.10.1002/9781118680605 ↗
46. What makes immersive virtual reality the ultimate empathy machine? Discerning the underlying mechanisms of change10.1016/j.chb.2020.106431 ↗
47. With New Eyes I See: Embodiment, empathy and silence in digital heritage interpretation10.1080/13527258.2017.1341946 ↗
48. Mann, S., and Fung, J. (2001, January 14–15). Videoorbits on eye tap devices for deliberately diminished reality or altering the visual perception of rigid planar patches of a real world scene. Proceedings of the International Symposium on Mixed Reality (ISMR2001), Yokohama, Japan.
49. A survey of diminished reality: Techniques for visually concealing, eliminating, and seeing through real objects
50. Diminished reality system with real-time object detection using deep learning for onsite landscape simulation during redevelopment10.1016/j.envsoft.2020.104759 ↗
51. Dhamo, H., Navab, N., and Tombari, F. (November, January 27). Object-driven multi-layer scene decomposition from a single image. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea.
52. Yeh, R.A., Chen, C., Lim, T.Y., Schwing, A.G., Hasegawa-Johnson, M., and Do, M.N. (2017, January 21–26). Semantic Image Inpainting with Deep Generative Models. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.728 ↗
53. Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., and Huang, T.S. (November, January 27). Free-Form Image Inpainting with Gated Convolution. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.10.1109/ICCV.2019.00457 ↗
54. Hartholt, A., Fast, E., Reilly, A., Whitcup, W., Liewer, M., and Mozgai, S. (2019, January 9–11). Ubiquitous virtual humans: A multi-platform framework for embodied AI agents in XR. Proceedings of the 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), San Diego, CA, USA.10.1109/AIVR46125.2019.00072 ↗
55. Ioannides, M., Magnenat-Thalmann, N., and Papagiannakis, G. (2017). Mixed Reality and Gamification for Cultural Heritage, Springer.10.1007/978-3-319-49607-8 ↗
56. Real Time Animation and Illumination in Ancient Roman Sites
57. Papaefthymiou, M., Feng, A., Shapiro, A., and Papagiannakis, G. (2015, January 2–6). A fast and robust pipeline for populating mobile AR scenes with gamified virtual characters. Proceedings of the SIGGRAPH Asia Mobile Graphics and Interactive Applications, Kobe, Japan.10.1145/2818427.2818463 ↗
58. Kasap, Z., and Magnenat-Thalmann, N. (2007). Intelligent virtual humans with autonomy and personality: State-of-the-art. Intell. Decis. Technol.10.3233/IDT-2007-11-202 ↗
59. Cai, Y., and See, S. (2015). Real-Time Separable Subsurface Scattering for Animated Virtual Characters. GPU Computing and Applications, Springer.10.1007/978-981-287-134-3 ↗
60. Fast deformable model-based human performance capture and FVV using consumer-grade RGB-D sensors10.1016/j.patcog.2018.02.013 ↗
61. Frueh, C., Sud, A., and Kwatra, V. (August, January 30). Headset removal for virtual and mixed reality. Proceedings of the ACM SIGGRAPH 2017 Talks on SIGGRAPH ’17, Los Angeles, CA, USA.10.1145/3084363.3085083 ↗
62. Sandor, C., Fuchs, M., Cassinelli, A., Li, H., Newcombe, R., Yamamoto, G., and Feiner, S. (2015). Breaking the Barriers to True Augmented Reality. arXiv.
63. The design of natural interaction10.1007/s11042-007-0190-z ↗
64. Evaluating the Impact of Highly Immersive Technologies and Natural Interaction on Player Engagement and Flow Experience in Games10.1007/978-3-319-24589-8_13 ↗
65. Stephanidis, C. (2012). Human Factors in Ambient Intelligence Environments. Handbook of Human Factors and Ergonomics, John Wiley & Sons.10.1002/9781118131350.ch49 ↗
66. Toward Interconnected Virtual Reality: Opportunities, Challenges, and Enablers10.1109/MCOM.2017.1601089 ↗
67. McTear, M., Callejas, Z., and Griol, D. (2016). Conversational Interfaces: Past and Present. The Conversational Interface, Springer.10.1007/978-3-319-32967-3 ↗
68. Otter, D.W., Medina, J.R., and Kalita, J.K. (2020). A survey of the usages of deep learning for natural language processing. IEEE Trans. Neural Netw. Learn. Syst., 1–21.
69. Dean, D., and Edson, G. (2013). Handbook for Museums, Routledge.10.4324/9780203552162 ↗
70. Beyond virtual museums: Experiencing immersive virtual reality in real museums10.1016/j.culher.2010.04.001 ↗
71. Jung, T., Tom Dieck, M.C., Lee, H., and Chung, N. (2016, January 2–5). Effects of Virtual Reality and Augmented Reality on Visitor Experiences in Museum. Proceedings of the Information and Communication Technologies in Tourism, Bilbao, Spain.10.1007/978-3-319-28231-2_45 ↗
72. Margetis, G., Papagiannakis, G., and Stephanidis, C. (2019). Realistic Natural Interaction with Virtual Statues in X-Reality Environments. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci., 801–808.10.5194/isprs-archives-XLII-2-W11-801-2019 ↗
73. Apostolakis, K.C., Margetis, G., and Stephanidis, C. (2020, January 9–13). Pleistocene Crete: A narrative, interactive mixed reality exhibition that brings prehistoric wildlife back to life. Proceedings of the 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Recife, Brazil.10.1109/ISMAR-Adjunct51615.2020.00069 ↗
74. Simulating natural history: Walking with Dinosaurs as hyper-real edutainment10.1080/09505430309019 ↗
75. Ancient DNA forces reconsideration of evolutionary history of Mediterranean pygmy elephantids10.1098/rsbl.2006.0467 ↗
76. A Pleistocene endemic island form within the genus Athene: Athene cretensis n. sp. (Aves, Strigiformes) from Crete
77. Pleistocene deer fauna in Crete: Its adaptive radiation and extinction10.3759/tropics.10.125 ↗
78. Evaluating visitor experience of digital interpretation and presentation technologies at cultural heritage sites: A case study of the old town, Zuoying