Encoding, Exchange and Manipulation of Captured Immersive VR Sessions for Learning Environments: the PRISMIN Framework
Source abstract
Abstract
Capturing immersive VR sessions performed by remote learners using head-mounted displays (HMDs) may provide valuable insights on their interaction patterns, virtual scene saliency and spatial analysis. Large collected records can be exploited as transferable data for learning assessment, detect unexpected interactions or fine-tune immersive VR environments. Within the online learning segment, the exchange of such records among different peers over the network presents several challenges related to data transport and/or its decoding routines. In the presented work, we investigate applications of an image-based encoding model and its implemented architecture to capture users’ interactions performed during VR sessions. We present the PRISMIN framework and how the underneath image-based encoding can be exploited to exchange and manipulate captured VR sessions, comparing it to existing approaches. Qualitative and quantitative results are presented in order to assess the encoding model and the developed open-source framework.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Image and Video Quality Assessment · Advanced Image and Video Retrieval Techniques · Visual Attention and Saliency Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
VHLab, CNR ISPC (Institute for Heritage Sciences)—Area della Ricerca Roma 1, Via Salaria Km. 29,300, 00015 Monterotondo, Italy
Department of Computer Science - Sapienza University, Via Salaria 113, 00185 Rome, Italy
References (41)
Show source-provided references
1. Maher, D. (2020). Altered Realities: How Virtual and Augmented Realities Are Supporting Learning. Handbook of Research on Innovative Pedagogies and Best Practices in Teacher Education, IGI Global.10.4018/978-1-5225-9232-7.ch003 ↗
2. Mantovani, G. (2003). VR learning: Potential and challenges for the use of 3D. Towards Cyberpsychology: Mind, Cognitions, and Society in the Internet Age, IOS Press.
3. Understanding the diffusion of virtual reality glasses: The role of media, fashion and technology10.1016/j.techfore.2018.09.008 ↗
4. Shute, V., Rahimi, S., and Emihovich, B. (2017). Assessment for Learning in Immersive Environments. Virtual, Augmented, and Mixed Realities in Education, Springer.10.1007/978-981-10-5490-7_5 ↗
5. Airey, D., and Benckendorff, P. (2017). Standards, benchmarks and assurance of learning. Handbook of Teaching and Learning in Tourism, Edward Elgar Publishing.10.4337/9781784714802.00048 ↗
6. Visual analytics10.1109/MCG.2004.39 ↗
7. Agus, M., Marton, F., Bettio, F., and Gobbetti, E. (2016, January 5–7). Interactive 3D exploration of a virtual sculpture collection: An analysis of user behavior in museum setting. Proceedings of the 13th Eurographics Worhshop on Graphics and Cultural Heritage, Genoa, Italy.
8. Discovering landmark preferences and movement patterns from photo postings10.1111/j.1467-9671.2010.01235.x ↗
9. Data clustering: 50 years beyond K-means10.1016/j.patrec.2009.09.011 ↗
10. Battersby, S.A., Lavelle, M., Healey, P.G., and McCabe, R. (2008, January 27). Analysing Interaction: A comparison of 2D and 3D techniques. Proceedings of the Programme of the Workshop on Multimodal Corpora, Marrakech, Morocco.
11. Boletsis, C. (2017). The new era of virtual reality locomotion: A systematic literature review of techniques and a proposed typology. Multimodal Technol. Interact., 1.10.3390/mti1040024 ↗
12. Delivering and using 3D models on the web: Are we ready?10.4995/var.2017.6405 ↗
13. Maclntyre, B., and Smith, T.F. (2018, January 16–20). Thoughts on the Future of WebXR and the Immersive Web. Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Munich, Germany.10.1109/ISMAR-Adjunct.2018.00099 ↗
14. See, Z.S., Lee, X.S., Brimo, A., Thwaites, H., and Goodman, L. (2018, January 16–17). MOOC for AR VR Training. Proceedings of the IEEE Games, Entertainment, Media Conference (GEM), Galway, Ireland.10.1109/GEM.2018.8516514 ↗
15. Saliency in VR: How do people explore virtual environments?10.1109/TVCG.2018.2793599 ↗
16. Upenik, E., and Ebrahimi, T. (2017, January 10–14). A simple method to obtain visual attention data in head mounted virtual reality. Proceedings of the 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Hong Kong, China.10.1109/ICMEW.2017.8026231 ↗
17. Knorr, S., Ozcinar, C., Fearghail, C.O., and Smolic, A. (2018, January 13–14). Director’s Cut-A Combined Dataset for Visual A ention Analysis in Cinematic VR Content. Proceedings of the CVMP ’18 15th ACM SIGGRAPH European Conference on Visual Media Production, London, UK.10.1145/3278471.3278472 ↗
18. Chandler, T., Cordeil, M., Czauderna, T., Dwyer, T., Glowacki, J., Goncu, C., Klapperstueck, M., Klein, K., Marriott, K., and Schreiber, F. (2015, January 22–25). Immersive analytics. Proceedings of the Big Data Visual Analytics (BDVA), Hobart, Australia.10.1109/BDVA.2015.7314296 ↗
19. Butcher, P.W., Roberts, J.C., and Ritsos, P.D. (2016, January 23–28). Immersive Analytics with WebVR and Google Cardboard. Proceedings of the Posters of IEEE VIS, Baltimaore, MD, USA.
20. Butcher, P.W., John, N.W., and Ritsos, P.D. (2018, January 21–26). Towards a Framework for Immersive Analytics on the Web. Proceedings of the IEEE Conference on Visualization: InfoVis, Berlin, Germany.
21. Hadjar, H., Meziane, A., Gherbi, R., Setitra, I., and Aouaa, N. (2018, January 8–10). WebVR based interactive visualization of open health data. Proceedings of the 2nd International Conference on Web Studies, Paris, France.10.1145/3240431.3240442 ↗
22. Image file formats: Past, present, and future10.1148/radiographics.21.3.g01ma25789 ↗
23. Fast, progressive loading of binary-encoded declarative-3d web content10.1109/MCG.2013.52 ↗
24. Dworak, D., and Pietruszka, M. (2015). Fast encoding of huge 3D data sets in lossless PNG format. New Research in Multimedia and Internet Systems, Springer.10.1007/978-3-319-10383-9_2 ↗
25. Geometry images10.1145/566654.566589 ↗
26. Fanini, B., and Cinque, L. (2019). An Image-Based Encoding to Record and Track Immersive VR Sessions. International Conference on Computational Science and Its Applications, Springer.10.1007/978-3-030-24296-1_25 ↗
27. Fanini, B., and Cinque, L. (2019). Encoding immersive sessions for online, interactive VR analytics. Virtual Real.10.1007/s10055-019-00405-w ↗
28. Manku, G.S., and Motwani, R. (2002, January 20–23). Approximate frequency counts over data streams. Proceedings of the VLDB’02, 28th International Conference on Very Large Databases, HongKong, China.10.1016/B978-155860869-6/50038-X ↗
29. Habgood, M.J., Moore, D., Wilson, D., and Alapont, S. (2018, January 18–22). Rapid, continuous movement between nodes as an accessible virtual reality locomotion technique. Proceedings of the 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Reutlingen, Germany.
30. ARIADNE: A research infrastructure for archaeology10.1145/3064527 ↗
31. Fanini, B., Pescarin, S., and Palombini, A. (2019). A cloud-based architecture for processing and dissemination of 3D landscapes online. Digit. Appl. Archaeol. Cult. Herit., e00100.10.1016/j.daach.2019.e00100 ↗
32. Antal, A., Bota, E., Ciongradi, C., D’Annibale, E., Demetrescu, E., Dima, C., Fanini, B., and Ferdani, D. (2016). A complete workflow from the data collection on the field to the deployment of a Virtual Museum: The case of Virtual Sarmizegetusa. Digit. Appl. Archaeol. Cult. Herit.
33. Barsanti, S.G., Malatesta, S.G., Lella, F., Fanini, B., Sala, F., Dodero, E., and Petacco, L. (2018). The winckelmann300 project: Dissemination of culture with virtual reality at the capitoline museum in rome. Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci., 42.
34. Cantelon, M., Harter, M., Holowaychuk, T., and Rajlich, N. (2014). Node js in Action, Manning Greenwich.
35. Node.js: Using JavaScript to build high-performance network programs10.1109/MIC.2010.145 ↗
36. Fanini, B., d’Annibale, E., Demetrescu, E., Ferdani, D., and Pagano, A. (October, January 28). Engaging and shared gesture-based interaction for museums the case study of K2R international expo in Rome. Proceedings of the 2015 Digital Heritage, Granada, Spain.10.1109/DigitalHeritage.2015.7413880 ↗
37. Antonaci, A., and Pagano, A. (2015, January 2–4). Technology enhanced visit to museums. A case study: Keys to Rome. Proceedings of the INTED2015, Madrid, Spain.
38. Liu, X., Wang, Y., Hu, Q., and Yu, D. (2016, January 4–6). A scan-line-based data compression approach for point clouds: Lossless and effective. Proceedings of the 2016 4th International Workshop on Earth Observation and Remote Sensing Applications (EORSA), Guangzhou, China.
39. Ferdani, D., Fanini, B., Piccioli, M.C., Carboni, F., and Vigliarolo, P. (2020). 3D reconstruction and validation of historical background for immersive VR applications and games: The case study of the Forum of Augustus in Rome. J. Cult. Herit.10.1016/j.culher.2019.12.004 ↗
40. Bozgeyikli, E., Raij, A., Katkoori, S., and Dubey, R. (2016, January 16–19). Point & teleport locomotion technique for virtual reality. Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play, Austin, TX, USA.10.1145/2967934.2968105 ↗
41. Head-Mounted Display Virtual Reality in Post-Secondary Education and Skill Training: A Systematic Review. Frontiers in Education