HybridFilm: A Mixed-Reality History Tool Enabling Interoperability Between Screen Space and Immersive Environments
Source abstract
Abstract
History tools facilitate iterative analysis data by allowing users to view, retrieve, and revisit visualization states. However, traditional history tools are constrained by screen space limitations, which restrict the user’s ability to fully understand historical states and make it challenging to provide an intuitive preview of these states. Most immersive history tools, in contrast, operate independently of screen space and fail to consider their integration. This paper proposes HybridFilm, an innovative mixed-reality history tool that seamlessly integrates screen space and immersive reality. First, it expands the user’s understanding of historical states through a multi-source spatial fusion approach. Second, it proposes a “focus + context”-based multi-source spatial historical data visualization and interaction scheme. Furthermore, we assessed the usability and utility of HybridFilm through experimental evaluation. In comparison to traditional history tools, HybridFilm offers a more intuitive and immersive experience while maintaining a comparable level of interaction comfort and fluency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Augmented Reality Applications · Virtual Reality Applications and Impacts · Interactive and Immersive Displays
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Science and Engineering, Yanshan University, Qinhuangdao 066004, China
Key Laboratory for Software Engineering of Hebei Province, Qinhuangdao 066004, China
Funding
National Science Foundation of China · Natural Science Foundation of Hebei Province · Innovation Capability Improvement Plan Project of Hebei Province
References (50)
Show source-provided references
1. Rupp, D., Kuhlen, T., and Weissker, T. (2023, January 16–20). TENETvr: Comprehensible Temporal Teleportation in Time-Varying Virtual Environments. Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Sydney, Australia.10.1109/ISMAR59233.2023.00108 ↗
2. OneTrace: Improving Event Recall and Coordination with Cross-Application Interaction Histories
3. (2025, June 18). Git. Git-Scm.com. Available online: https://git-scm.com/.
4. Nonlinear revision control for images10.1145/2010324.1965000 ↗
5. Zhang, L., Agrawal, A., Oney, S., and Guo, A. (2023, January 23–28). Vrgit: A version control system for collaborative content creation in virtual reality. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany.10.1145/3544548.3581136 ↗
6. Sauermann, L., Bernardi, A., and Dengel, A. (2005, January 6). Overview and Outlook on the Semantic Desktop. Proceedings of the Semantic Desktop Workshop, Galway, Irelan.
7. Graphical histories for visualization: Supporting analysis, communication, and evaluation10.1109/TVCG.2008.137 ↗
8. Hoff, A., Seidl, C., and Lanza, M. (2024, January 12–15). Immersive Software Archaeology: Exploring Software Architecture and Design in Virtual Reality. Proceedings of the 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) IEEE, Rovaniemi, Finland.10.1109/SANER60148.2024.00011 ↗
9. Mehra, R., Sharma, V.S., and Vikrant Kaulgud Sanjay Podder Burden, A.P. (2020, January 21–25). Towards immersive comprehension of software systems using augmented reality. Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, Melbourne, Australia.10.1145/3324884.3418907 ↗
10. Kreber, L., Diehl, S., and Weil, P. (2022, January 3–4). Idevelopar: A programming interface to enhance code understanding in augmented reality. Proceedings of the 2022 Working Conference on Software Visualization (VISSOFT), Limassol, Cyprus.10.1109/VISSOFT55257.2022.00017 ↗
11. Merino, L., Hess, M., Bergel, A., Nierstrasz, O., and Weiskopf, D. (2019). Perfvis: Pervasive Visualization in Immersive Augmented Reality for Performance Awareness. arXiv.10.1145/3302541.3313104 ↗
12. Waller, J., Wulf, C., Fittkau, F., Döhring, P., and Hasselbring, W. (2013, January 27–28). Synchrovis: 3d visualization of monitoring traces in the city metaphor for analyzing concurrency. Proceedings of the 2013 First IEEE Working Conference on Software Visualization (VISSOFT), Eindhoven, The Netherlands.10.1109/VISSOFT.2013.6650520 ↗
13. Sharma, V.S., Mehra, R., Kaulgud, V., and Podder, S. (2019, January 27). An extended reality approach for creating immersive software project workspaces. Proceedings of the 2019 IEEE/ACM 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE), Montreal, QC, Canada.10.1109/CHASE.2019.00013 ↗
14. Lee, B., Cordeil, M., Prouzeau, A., Jenny, B., and Dwyer, T. (May, January 29). A design space for data visualisation transformations between 2d and 3d in mixed-reality environments. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA.10.1145/3491102.3501859 ↗
15. Cavallo, M., Dolakia, M., Havlena, M., Ocheltree, K., and Podlaseck, M. (2019, January 12–15). Immersive insights: A hybrid analytics system forcollaborative exploratory data analysis. Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology, Parramatta, NSW, Australia.10.1145/3359996.3364242 ↗
16. Myers, B.A. (2024, January 11–16). Interaction Techniques–History, Design and Evaluation. Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3613905.3636276 ↗
17. Seraji, M.R., and Stuerzlinger, W. (2022, January 17–21). Hybridaxes: An immersive analytics tool with interoperability between 2d and immersive reality modes. Proceedings of the 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Singapore.10.1109/ISMAR-Adjunct57072.2022.00036 ↗
18. Chen, H.T., Wei, L.Y., Hartmann, B., and Agrawala, M. (2016, January 26–28). Data-driven adaptive history for image editing. Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, Redmond, WA, USA.10.1145/2856400.2856417 ↗
19. Doboš, J., and Steed, A. (2012, January 4). 3D revision control framework. Proceedings of the 17th International Conference on 3D Web Technology, New York, NY, USA.10.1145/2338714.2338736 ↗
20. Lilija, K., Pohl, H., and Hornbæk, K. (2020, January 25–30). Who put that there? temporal navigation of spatial recordings by direct manipulation. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3313831.3376604 ↗
21. Myers, B.A., Lai, A., Le, T.M., Yoon, Y., Faulring, A., and Brandt, J. (2015, January 18–23). Selective undo support for painting applications. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Seoul, Republic of Korea.10.1145/2702123.2702543 ↗
22. Lyra 2: Designing interactive visualizations by demonstration10.1109/TVCG.2020.3030367 ↗
23. Meshhisto: Collaborative modeling by sharing and retargeting editing histories10.1145/2816795.2818110 ↗
24. Liu, Y., Liao, S., Jin, Y., Ma, M., and Tang, W. (2024, January 24–26). Embodied Cognition and MR-Based Interactive Narrative Design: The Case of ‘Encountering Sanmao’ at the Former Residence of Zhang Leping. Proceedings of the 2024 10th International Conference on Virtual Reality (ICVR), Bournemouth, UK.10.1109/ICVR62393.2024.10869102 ↗
25. Comparative study of HMD-based virtual and augmented realities for immersive museums: User acceptance, medium, and learning
26. Wang, X., Besançon, L., Rousseau, D., Sereno, M., Ammi, M., and Isenberg, T. (2020, January 25–30). Towards an understanding of augmented reality extensions for existing 3D data analysis tools. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3313831.3376657 ↗
27. A hybrid 2D/3D user interface for radiological diagnosis10.1007/s10278-017-0002-6 ↗
28. Zhu-Tian, C., Tong, W., Wang, Q., Bach, B., and Qu, H. (2020, January 25–30). Augmenting static visualizations with paparvis designer. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3313831.3376436 ↗
29. Fröhler, B., Anthes, C., Pointecker, F., Friedl, J., Schwajda, D., Riegler, A., Tripathi, S., Holzmann, C., Brunner, M., and Jodlbauer, H. (2022, January 08). A Survey on Cross-Virtuality Analytics. Proceedings of the Computer Graphics Forum Hoboken, New Jersey, USA.10.1111/cgf.14447 ↗
30. Zhu, F., and Grossman, T. (2020, January 25–30). Bishare: Exploring bidirectional interactions between smartphones and head-mounted augmented reality. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3313831.3376233 ↗
31. Langner, R., Satkowski, M., Büschel, W., and Dachselt, R. (2021, January 8–13). Marvis: Combining mobile devices and augmented reality for visual data analysis. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Online.10.1145/3411764.3445593 ↗
32. Personal augmented reality for information visualization on large interactive displays10.1109/TVCG.2020.3030460 ↗
33. Effects of visual realism on avatar perception in immersive and non-immersive virtual environments
34. Enhancing museum experiences: Using immersive environments to evaluate soundscape preferences10.1121/10.0035832 ↗
35. Bowman, D.A., and Wingrave, C.A. (2001, January 13–17). Design and evaluation of menu systems for immersive virtual environments. Proceedings of the IEEE Virtual Reality 2001 IEEE, Yokohama, Japan.
36. Reiter, K., Pfeuffer, K., Esteves, A., Mittermeier, T., and Alt, F. (2022, January 8–11). Look & turn: One-handed and expressive menu interaction by gaze and arm turns in vr. Proceedings of the 2022 Symposium on Eye Tracking Research and Applications, Seattle, WA, USA.10.1145/3517031.3529233 ↗
37. Chen, X., Guo, D., Feng, L., Chen, B., and Liu, W. (2023, January 16–20). Compass+ Ring: A Multimodal Menu to Improve Interaction Performance and Comfortability in One-handed Scenarios. Proceedings of the 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Sydney, Australia.10.1109/ISMAR59233.2023.00062 ↗
38. Lindlbauer, D., Feit, A.M., and Hilliges, O. (2019, January 20–23). Context-aware online adaptation of mixed reality interfaces. Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, New Orleans, LA, USA.10.1145/3332165.3347945 ↗
39. Cheng, Y.F., Luong, T., Fender, A.R., Streli, P., and Holz, C. (2022, January 17–21). ComforTable user interfaces: Surfaces reduce input error, time, and exertion for tabletop and mid-air user interfaces. Proceedings of the 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Singapore.10.1109/ISMAR55827.2022.00029 ↗
40. Satriadi, K.A., Ens, B., Cordeil, M., Jenny, B., Czauderna, T., and Willett, W. (2019, January 23–27). Augmented reality map navigation with freehand gestures. Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan.10.1109/VR.2019.8798340 ↗
41. Wentzel, J., Lakier, M., Hartmann, J., and Falah Shazib Géry Casiez Vogel, D. (2024). A Comparison of Virtual Reality Menu Archetypes: Raycasting, Direct Input, and Marking Menus. IEEE Trans. Vis. Comput. Graph., 1–15.10.1109/TVCG.2024.3420236 ↗
42. Li, Y., Fischer, F., Dwyer, T., Ens, B., Crowther, R., Kristensson, P.O., and Tag, B. (May, January 26). Alphapig: The nicest way to prolong interactive gestures in extended reality. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, Yokohama Japan.10.1145/3706598.3714249 ↗
43. Evaluation of a hybrid of hand gesture and controller inputs in virtual reality10.1080/10447318.2020.1809248 ↗
44. Wu, S., Byrne, D., and Steenson, M.W. (2020, January 25–30). “Megereality”: Leveraging Physical Affordances for Multi-Device Gestural Interaction in Augmented Reality. Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3334480.3383170 ↗
45. (2025, June 18). Mixed Reality Toolkit (MRTK) 2.8.0. 2023. Microsoft. Available online: https://github.com/microsoft/MixedRealityToolkit-Unity/releases/tag/v2.8.0.
46. (2024, July 13). Unity 2022.3.15. Unity Technologies. Available online: https://unity.com/releases/editor/whats-new/2022.3.15.10.3126/unityj.v3i01.43310 ↗
47. Robinson, A.C., and Weaver, C. (2006, January 20). Re-visualization: Interactive visualization of the process of visual analysis. Proceedings of the Workshop on Visualization, Analytics & Spatial Decision Support at the GIScience Conference, Muenster, Germany.
48. Dube, T.J., and Arif, A.S. (2019, January 26–31). Text entry in virtual reality: A comprehensive review of the literature. Proceedings of the Human-Computer Interaction. Recognition and Interaction Technologies: Thematic Area, HCI 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA. Proceedings, Part II 21.
49. He, Y., Hu, Y., Feng, H., Li, C., and Shen, X. (2021, January 16–17). Comparative Analysis of 3D Interactive Modes in Different Object Layouts in Mixed Reality. Proceedings of the Ninth International Symposium of Chinese CHI, Online.10.1145/3490355.3490371 ↗
50. Likert Scale: Explored and Explained10.9734/BJAST/2015/14975 ↗