Shared Presence via XR Communication and Interaction Within a Dynamically Updated Digital Twin of a Smart Space: Conceptual Framework and Research Challenges
Source abstract
Abstract
The integration of emerging eXtended Reality (XR) technologies, digital twins (DTs), smart environments, and advanced mobile and wireless networks is set to enable novel forms of immersive interaction and communication. This paper proposes a high-level conceptual framework for shared presence via XR-based communication and interaction within a virtual reality (VR) representation of the digital twin of a smart space. The digital twin is continuously updated and synchronized—both spatially and temporally—with a physical smart space equipped with sensors and actuators. This architecture enables interactive experiences and fosters a sense of co-presence between a local user in the smart physical environment utilizing augmented reality (AR) and a remote VR user engaging through the digital counterpart. We present our lab deployment architecture used as a basis for ongoing experimental work related to testing and integrating functionalities defined in the conceptual framework. Finally, key technology requirements and research challenges are outlined, aiming to provide a foundation for future research efforts in immersive, interconnected XR systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Virtual Reality Applications and Impacts · Augmented Reality Applications · IoT and Edge/Fog Computing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, 10000 Zagreb, Croatia
Delta Reality, Radnicka Cesta 41, 10000 Zagreb, Croatia
Ericsson Nikola Tesla d.d., Krapinska 45, 10000 Zagreb, Croatia
Funding
European Union—NextGenerationEU
References (56)
Show source-provided references
1. New European Media (NEM) (2024, November 28). NEM: List of Topics for the Work Program 2023–2024. Available online: https://nem-initiative.org/wp-content/uploads/2022/05/nem-list-of-topics-for-the-work-program-2023-2024.pdf?x46896.
2. Empowering Digital Twins with Extended Reality Collaborations10.1016/j.vrih.2022.06.004 ↗
3. Künz, A., Rosmann, S., Loria, E., and Pirker, J. (2022, January 12–16). The Potential of Augmented Reality for Digital Twins: A Literature Review. Proceedings of the 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Christchurch, New Zealand.10.1109/VR51125.2022.00058 ↗
4. What is XR? Towards a framework for augmented and virtual reality10.1016/j.chb.2022.107289 ↗
5. Skorin-Kapov, L., Matijasevic, M., Podnar Zarko, I., Kusek, M., Huljenic, D., Skarica, V., Skarica, D., and Grguric, A. (2025, January 3–6). DIGIPHY: XR Communication and Interaction within a Dynamically Updated Digital Twin of a Smart Space, 2025. Proceedings of the EUCNC & 6G Summit, Poznań, Poland. Available online: https://tinyurl.com/2y4dvcdy.
6. Florez-Revuelta, F., Ake-Kob, A., Climent-Perez, P., Coelho, P., Colonna, L., Dahabiyeh, L., Dantas, C., Dogru-Huzmeli, E., Ekenel, H.K., and Jevremovic, A. (2024). 50 Questions on Active Assisted Living Technologies, Global ed., Zenodo.
7. Brzica, L., Matanović, F., Vlahović, S., Pavlin Bernardić, N., and Skorin-Kapov, L. (April, January 31). Analysis of User Experience and Task Performance in a Multi-User Cross-Reality Virtual Object Manipulation Task. Proceedings of the 17th International Workshop on IMmersive Mixed and Virtual Environment Systems, Stellenbosch, South Africa.
8. Oppermann, L., Buchholz, F., and Uzun, Y. (2023, January 23–28). Industrial Metaverse: Supporting remote maintenance with avatars and digital twins in collaborative XR environments. Proceedings of the Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany.10.1145/3544549.3585835 ↗
9. Bormann, C., Ersue, M., and Keränen, A. (2014). Terminology for Constrained-Node Networks, RFC Editor. RFC 7228.10.17487/rfc7228 ↗
10. Nerf: Representing scenes as neural radiance fields for view synthesis10.1145/3503250 ↗
11. Gao, K., Gao, Y., He, H., Lu, D., Xu, L., and Li, J. (2022). NeRF: Neural Radiance Field in 3D Vision, a Comprehensive Review. arXiv.
12. Lindell, D.B., Martel, J.N., and Wetzstein, G. (2021, January 20–25). Autoint: Automatic integration for fast neural volume rendering. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01432 ↗
13. Reiser, C., Peng, S., Liao, Y., and Geiger, A. (2021, January 11–17). Kilonerf: Speeding up neural radiance fields with thousands of tiny MLPs. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.01407 ↗
14. 3D Gaussian Splatting for Real-Time Radiance Field Rendering10.1145/3592433 ↗
15. Chen, G., and Wang, W. (2024). A survey on 3D Gaussian Splatting. arXiv.
16. Haslbauer, P., Pullen, M., Praprotnik, M., Stimec, S., Hospital, M., Grosbois, A., Reichherzer, C., and Smolic, A. (2025, January 9–11). VR Planica: Gaussian Splatting Workflows for Immersive Storytelling. Proceedings of the ACM International Conference on Interactive Media Experiences Workshops, Porto Alegre, RS, Brasil.10.5753/imxw.2025.8141 ↗
17. Jiang, Y., Yu, C., Xie, T., Li, X., Feng, Y., Wang, H., Li, M., Lau, H., Gao, F., and Yang, Y. (August, January 27). VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality. Proceedings of the ACM SIGGRAPH 2024 Conference Papers, Denver, CO, USA.10.1145/3641519.3657448 ↗
18. (2025, June 21). Aras Pranckevičius. Available online: https://github.com/aras-p/UnityGaussianSplatting.
19. Zheng, S., Zhou, B., Shao, R., Liu, B., Zhang, S., Nie, L., and Liu, Y. (2024, January 16–22). Gps–Gaussian: Generalizable pixel-wise 3D Gaussian Splatting for real-time human novel view synthesis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.01861 ↗
20. Georgs: Geometric regularization for real-time novel view synthesis from sparse inputs10.1109/TCSVT.2024.3447113 ↗
21. Ren, K., Jiang, L., Lu, T., Yu, M., Xu, L., Ni, Z., and Dai, B. (2024). Octree-GS: Towards consistent Real-Time Rendering with LOD-Structured 3D Gaussians. arXiv.10.1109/TPAMI.2025.3568201 ↗
22. Lee, J.C., Rho, D., Sun, X., Ko, J.H., and Park, E. (2024, January 16–22). Compact 3D Gaussian representation for radiance field. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.02052 ↗
23. Niedermayr, S., Stumpfegger, J., and Westermann, R. (2024, January 17–21). Compressed 3D Gaussian Splatting for accelerated novel view synthesis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.00985 ↗
24. Wu, G., Yi, T., Fang, J., Xie, L., Zhang, X., Wei, W., Liu, W., Tian, Q., and Wang, X. (2024, January 17–21). 4D Gaussian Splatting for real-time dynamic scene rendering. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.01920 ↗
25. Schieber, H., Young, J., Langlotz, T., Zollmann, S., and Roth, D. (2025, January 8–12). Semantics-controlled gaussian splatting for outdoor scene reconstruction and rendering in virtual reality. Proceedings of the 2025 IEEE Conference Virtual Reality and 3D User Interfaces (IEEE VR), Saint-Malo, France.10.1109/VR59515.2025.00056 ↗
26. Fei, B., Xu, J., Zhang, R., Zhou, Q., Yang, W., and He, Y. (2024). 3D Gaussian Splatting as New Era: A Survey. IEEE Trans. Vis. Comput. Graph., 1–20. early access.10.1109/TVCG.2024.3397828 ↗
27. 3D Gaussian Splatting: Survey, technologies, challenges, and opportunities10.1109/TCSVT.2025.3538684 ↗
28. Review on SLAM algorithms for Augmented Reality10.1016/j.displa.2024.102806 ↗
29. Camera localization for augmented reality and indoor positioning: A vision-based 3D feature database approach10.1080/17538947.2018.1564379 ↗
30. Jo, D., and Kim, G.J. (2019). AR Enabled IoT for a Smart and Interactive Environment: A Survey and Future Directions. Sensors, 19.10.3390/s19194330 ↗
31. Podnar Žarko, I., Mueller, S., Plociennik, M., and Rajtar, T. (2019, January 17–21). The symbIoTe Solution for Semantic and Syntactic Interoperability of Cloud-based IoT Platforms. Proceedings of the 2019 IEEE Global IoT Summit (GIoTS), Aarhus, Denmark.10.1109/GIOTS.2019.8766420 ↗
32. 6G networks: Beyond Shannon towards semantic and goal-oriented communications10.1016/j.comnet.2021.107930 ↗
33. An open framework for analyzing and modeling XR network traffic10.1109/ACCESS.2021.3113162 ↗
34. Realizing the tactile Internet: Haptic communications over next generation 5G cellular networks10.1109/MWC.2016.1500157RP ↗
35. Immersive interconnected virtual and augmented reality: A 5G and IoT perspective10.1007/s10922-020-09545-w ↗
36. Yeregui, I., Mejías, D., Pacho, G., Viola, R., Astorga, J., and Montagud, M. (2024). Edge Rendering Architecture for multiuser XR Experiences and E2E Performance Assessment. arXiv.10.1109/BMSB62888.2024.10608249 ↗
37. Cloud-based XR services: A survey on relevant challenges and enabling technologies
38. (2022). Quality of Experience (QoE) Requirements for Real-Time Multimedia Services over 5G Networks (Standard No. ITU-T GSTR-5GQoE). ITU-T Technical Report.
39. (2023). Technical Specification Group Services and System Aspects; Extended Reality (XR) in 5G (Release 18) (Standard No. 3GPP TR 26.928). Technical report; 3rd Generation Partnership Project; Version 18.0.0.
40. (2023). Technical Specification Group Radio Access Network; NR; Study on XR enhancements for NR (Release 18) (Standard No. 3GPP TR 38.835). Technical report; 3rd Generation Partnership Project; Version 18.0.1.
41. Toward Interactive Multi-User Extended Reality Using Millimeter-Wave Networking10.1109/MCOM.001.2300804 ↗
42. Toward supporting XR services: Architecture and enablers10.1109/JIOT.2022.3222103 ↗
43. The Roadmap of Communication and Networking in 6G for the Metaverse10.1109/MWC.019.2100721 ↗
44. 5G-Advanced toward 6G: Past, Present, and Future10.1109/JSAC.2023.3274037 ↗
45. Towards Immersive Digiphysical Experiences
46. Pérez, P., Gonzalez-Sosa, E., Gutiérrez, J., and García, N. (2022). Emerging immersive communication systems: Overview, taxonomy, and good practices for QoE assessment. Front. Signal Process., 2.10.3389/frsip.2022.917684 ↗
47. (2022). Quality of Experience Assessment of Extended Reality Meetings (Standard No. ITU-T P.1320). ITU-T Recommendation.
48. (2024). First Draft of P.IXC ‘Interactive Test Methods for Subjective Assessment of Extended Reality Communications’ (Standard No. ITU-T P.IXC). ITU-T SG12 Contribution 199.
49. Efficient VR-AR communication method using virtual replicas in XR remote collaboration10.1016/j.ijhcs.2024.103304 ↗
50. Sivelle, C., Palma, D., and De Moor, K. (2025, January 8–12). Security and Privacy for VR in non-entertainment sectors: A practice-based study of the challenges, strategies and gaps. Proceedings of the 2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Saint-Malo, France.10.1109/VRW66409.2025.00069 ↗
51. Extended privacy for extended reality: XR technology has 99 problems and privacy is several of them
52. McGill, and Mark (2021). The IEEE Global Initiative on Ethics of Extended Reality (XR) Report–Extended Reality (XR) and the Erosion of Anonymity and Privacy. Extended Reality (XR) and the Erosion of Anonymity and Privacy—White Paper, IEEE.
53. Privacy-enhancing technology and everyday augmented reality: Understanding bystanders’ varying needs for awareness and consent
54. Florez-Revuelta, F. (2024). Proceedings of the Joint visuAAL-GoodBrother Conference on Trustworthy Video- and Audio-Based Assistive Technologies, Zenodo.
55. IEEE Social Implications of Technology Standards Committee (SSIT/SC) (2025, June 21). IEEE P7016 Standard for Ethically Aligned Design and Operation of Metaverse Systems. Available online: https://standards.ieee.org/ieee/7016/11178/.
56. Paladin, M., Brzica, L., Matanovic, F., Kljajic, D., and Skorin-Kapov, L. (October, January 30). QoE and Task Performance Assessment of Mobile Robot Teleoperation via a VR-based Interface. Proceedings of the 17th International Conference on Quality of Multimedia Experience (Accepted for Publication), Madrid, Spain.