Application of Neural Radiance Fields (NeRFs) for 3D Model Representation in the Industrial Metaverse
Source abstract
Abstract
This study explores the utilization of Neural Radiance Fields (NeRFs), with a specific focus on the Instant NeRFs technique. The objective is to represent three-dimensional (3D) models within the context of the industrial metaverse, aiming to achieve a high-fidelity reconstruction of objects in virtual environments. NeRFs, renowned for their innovative approach, enable comprehensive model reconstructions by integrating diverse viewpoints and lighting conditions. The study employs tools such as Unity, Photon Pun2, and Oculus Interaction SDK to develop an immersive metaverse. Within this virtual industrial environment, users encounter numerous interactive six-dimensional (6D) models, fostering active engagement and enriching the overall experience. While initial implementations showcase promising results, they also introduce computational complexities. Nevertheless, this integration forms the basis for immersive comprehension and collaborative interactions within the industrial metaverse. The evolving potential of NeRF technology promises even more exciting prospects in the future.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: 3D Surveying and Cultural Heritage · 3D Shape Modeling and Analysis · Virtual Reality Applications and Impacts
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Instituto de Diseño y Fabricación, Universitat Politècnica de València, 46022 València, Spain
Funding
Spanish Government
References (35)
Show source-provided references
1. Effects of exposure to immersive computer-generated virtual nature and control environments on affect and cognition10.1038/s41598-022-26750-6 ↗
2. Umair, M., Sharafat, A., Lee, D.E., and Seo, J. (2022). Impact of Virtual Reality-Based Design Review System on User’s Performance and Cognitive Behavior for Building Design Review Tasks. Appl. Sci., 12.10.3390/app12147249 ↗
3. Solanes, J.E., Montava-Jordà, S., Golf-Laville, E., Colomer-Romero, V., Gracia, L., and Muñoz, A. (2023). Enhancing STEM Education through Interactive Metaverses: A Case Study and Methodological Framework. Appl. Sci., 13.10.3390/app131910785 ↗
4. Metaverse beyond the hype: Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy10.1016/j.ijinfomgt.2022.102542 ↗
5. Bhattacharya, P., Saraswat, D., Savaliya, D., Sanghavi, S., Verma, A., Sakariya, V., Tanwar, S., Sharma, R., Raboaca, M.S., and Manea, D.L. (2023). Towards future internet: The metaverse perspective for diverse industrial applications. Mathematics, 11.10.3390/math11040941 ↗
6. QoE enhancement of the industrial metaverse based on Mixed Reality application optimization10.1016/j.displa.2023.102463 ↗
7. Kontkanen, I. (2024, February 19). Using 3D Techniques in Video Game Concept Art: Creating Environmental Concept Art for a Video Game. BSc Thesis. Tampere University of Applied Sciences (TAMK). May 2021. Available online: https://www.theseus.fi/bitstream/handle/10024/501432/Iida_Kontkanen.pdf.
8. Ortiz, J.S., Guevara, B.S., Espinosa, E.G., Santana, J., Tamayo, L.R., and Andaluz, V.H. (2020, January 24–27). 3D Virtual Content for Education Applications. Proceedings of the 2020 15th Iberian Conference on Information Systems and Technologies (CISTI), Seville, Spain.10.23919/CISTI49556.2020.9140822 ↗
9. 3D technology to support teaching and learning in health care education—A scoping review10.1016/j.ijer.2020.101699 ↗
10. Montusiewicz, J., Barszcz, M., and Korga, S. (2022). Preparation of 3D Models of Cultural Heritage Objects to Be Recognised by Touch by the Blind-Case Studies. Appl. Sci., 12.10.3390/app122311910 ↗
11. Image-based 3D modelling: A review10.1111/j.1477-9730.2006.00383.x ↗
12. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis10.1145/3503250 ↗
13. Pumarola, A., Corona, E., Pons-Moll, G., and Moreno-Noguer, F. (2021, January 20–25). D-nerf: Neural radiance fields for dynamic scenes. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01018 ↗
14. Xu, Q., Xu, Z., Philip, J., Bi, S., Shu, Z., Sunkavalli, K., and Neumann, U. (2022, January 18–24). Point-nerf: Point-based neural radiance fields. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00536 ↗
15. Unity (2024, February 19). Game Development Platform. Available online: https://unity.com/.
16. (2024, February 19). UnrealEngine. Available online: https://www.unrealengine.com.
17. Quest, M. (2024, February 19). Oculus SDK Download. Available online: https://developer.oculus.com/downloads/package/oculus-platform-sdk.
18. Engine, P. (2024, February 19). Photon Unity Networking 2 Download. Available online: https://www.photonengine.com/pun.
19. Engine, P. (2024, February 19). Photon Voice Download. Available online: https://www.photonengine.com/voice.
20. NVlabs (2024, February 19). Instant Neural Graphics Primitives. Available online: https://github.com/NVlabs/instant-ngp.
21. (2024, February 19). Colmap. Available online: https://colmap.github.io/install.html.
22. (2024, February 19). ImageMagick Studio LLC. Available online: https://imagemagick.org.
23. (2024, February 19). kwea123. volumeShad2.shader. Software Resource. Available online: https://github.com/MistyNight/NerfUnity/commit/918ecd1bcb00b4df6cb1fa48678505ce95ce9bec.
24. (2024, February 22). RealityCapture—3D Models from Photos and/or Laser Scans. Available online: https://www.capturingreality.com/.
25. Tancik, M., Weber, E., Ng, E., Li, R., Yi, B., Wang, T., Kristoffersen, A., Austin, J., Salahi, K., and Ahuja, A. (2023, January 6–10). Nerfstudio: A Modular Framework for Neural Radiance Field Development. Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Proceedings, Los Angeles CA USA.10.1145/3588432.3591516 ↗
26. (2024, February 22). Luma AI. Available online: https://lumalabs.ai/.
27. Li, C., Li, S., Zhao, Y., Zhu, W., and Lin, Y. (November, January 30). RT-NeRF: Real-Time On-Device Neural Radiance Fields Towards Immersive AR/VR Rendering. Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design, San Diego, CA, USA.
28. (2024, February 22). Volinga. Available online: https://volinga.ai/.
29. (2024, February 22). GPU Benchmarks for Deep Learning|Lambda. Available online: https://lambdalabs.com/gpu-benchmarks.
30. Liu, S., Zhang, X., Zhang, Z., Zhang, R., Zhu, J.Y., and Russell, B.C. (2021, January 11–17). Editing Conditional Radiance Fields. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada.10.1109/ICCV48922.2021.00572 ↗
31. Kobayashi, S., Matsumoto, E., and Sitzmann, V. (2022). Decomposing NeRF for Editing via Feature Field Distillation. arXiv.
32. Mirzaei, A., Aumentado-Armstrong, T., Derpanis, K., Kelly, J., Brubaker, M.A., Gilitschenski, I., and Levinshtein, A. (2022). SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields. arXiv.10.1109/CVPR52729.2023.01980 ↗
33. Lan, Y., Loy, C.C., and Dai, B. (2022). Correspondence Distillation from NeRF-based GAN. arXiv.10.1007/s11263-023-01903-w ↗
34. Wu, Q., Tan, J., and Xu, K. (2022). PaletteNeRF: Palette-based Color Editing for NeRFs. arXiv.
35. Hedman, P., Srinivasan, P.P., Mildenhall, B., Barron, J., and Debevec, P. (2021, January 11–17). Baking Neural Radiance Fields for Real-Time View Synthesis. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, BC, Canada.10.1109/ICCV48922.2021.00582 ↗