Bi-Resolution Hash Encoding in Neural Radiance Fields: A Method for Accelerated Pose Optimization and Enhanced Reconstruction Efficiency
Source abstract
Abstract
NeRF has garnered extensive attention from researchers due to its impressive performance in three-dimensional scene reconstruction and realistic rendering. It is perceived as a potential pivotal technology for scene reconstruction in fields such as virtual reality and augmented reality. However, most NeRF-related research and applications heavily rely on precise pose data. The challenge of effectively reconstructing scenes in situations with inaccurate or missing pose data remains pressing. To address this issue, we examine the relationship between different resolution encodings and pose estimation and introduce BiResNeRF, a scene reconstruction method based on both low and high-resolution hash encoding modules, accompanied by a two-stage training strategy. The training strategy includes setting different learning rates and sampling strategies for different stages, designing stage transition signals, and implementing a smooth warm-up learning rate scheduling strategy after the phase transition. The experimental results indicate that our method not only ensures high synthesis quality but also reduces training time. Compared to other algorithms that jointly optimize pose, our training process is sped up by at least 1.3×. In conclusion, our approach efficiently reconstructs scenes under inaccurate poses and offers fresh perspectives and methodologies for pose optimization research in NeRF.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Vision and Imaging · Robotics and Sensor-Based Localization · 3D Shape Modeling and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Guizhou Key Laboratory of Information and Computing Science, Guizhou Normal University, Guiyang 550001, China
School of Mathematical Science, Guizhou Normal University, Guiyang 550001, China
Funding
Key Laboratory of Information and Computing Science Guizhou Province of Guizhou Normal University
References (34)
Show source-provided references
1. Xu, L., Xiangli, Y., Peng, S., Pan, X., Zhao, N., Theobalt, C., Dai, B., and Lin, D. (2023, January 18–22). Grid-guided Neural Radiance Fields for Large Urban Scenes. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00802 ↗
2. Tancik, M., Casser, V., Yan, X., Pradhan, S., Mildenhall, B., Srinivasan, P.P., Barron, J.T., and Kretzschmar, H. (2022, January 18–24). Block-nerf: Scalable large scene neural view synthesis. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00807 ↗
3. Yang, Z., Chen, Y., Wang, J., Manivasagam, S., Ma, W.C., Yang, A.J., and Urtasun, R. (2023, January 18–22). UniSim: A Neural Closed-Loop Sensor Simulator. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00140 ↗
4. Yang, J., Ivanovic, B., Litany, O., Weng, X., Kim, S.W., Li, B., Che, T., Xu, D., Fidler, S., and Pavone, M. (2023). EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision. arXiv.
5. Guo, Y., Chen, K., Liang, S., Liu, Y.J., Bao, H., and Zhang, J. (2021, January 11–17). Ad-nerf: Audio driven neural radiance fields for talking head synthesis. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00573 ↗
6. Gafni, G., Thies, J., Zollhofer, M., and Nießner, M. (2021, January 20–25). Dynamic neural radiance fields for monocular 4d facial avatar reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00854 ↗
7. Peng, S., Dong, J., Wang, Q., Zhang, S., Shuai, Q., Zhou, X., and Bao, H. (2021, January 11–17). Animatable neural radiance fields for modeling dynamic human bodies. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.01405 ↗
8. Fov-nerf: Foveated neural radiance fields for virtual reality10.1109/TVCG.2022.3203102 ↗
9. Rojas, S., Zarzar, J., Pérez, J.C., Sanakoyeu, A., Thabet, A., Pumarola, A., and Ghanem, B. (2023, January 2–3). Re-ReND: Real-time Rendering of NeRFs across Devices. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00336 ↗
10. Virtual reality and augmented reality—Emerging screening and diagnostic techniques in ophthalmology: A systematic review10.1016/j.survophthal.2022.02.001 ↗
11. Comparison of the effectiveness of the virtual simulator and video-assisted teaching on intravenous catheter insertion skills and self-confidence: A quasi-experimental study10.1016/j.nedt.2020.104596 ↗
12. The assessment of virtual reality training in antromastoidectomy simulation10.1007/s10055-021-00516-3 ↗
13. Virtual reality job interview training in adults with autism spectrum disorder10.1007/s10803-014-2113-y ↗
14. Sun, J., Shen, Z., Wang, Y., Bao, H., and Zhou, X. (2021, January 20–25). LoFTR: Detector-free local feature matching with transformers. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00881 ↗
15. ORB-SLAM: A versatile and accurate monocular SLAM system10.1109/TRO.2015.2463671 ↗
16. Distinctive image features from scale-invariant keypoints10.1023/B:VISI.0000029664.99615.94 ↗
17. Nerf: Representing scenes as neural radiance fields for view synthesis10.1145/3503250 ↗
18. Schonberger, J.L., and Frahm, J.M. (2016, January 27–30). Structure-from-motion revisited. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.445 ↗
19. Zhang, K., Riegler, G., Snavely, N., and Koltun, V. (2020). Nerf++: Analyzing and improving neural radiance fields. arXiv.
20. Martin-Brualla, R., Radwan, N., Sajjadi, M.S., Barron, J.T., Dosovitskiy, A., and Duckworth, D. (2021, January 20–25). Nerf in the wild: Neural radiance fields for unconstrained photo collections. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00713 ↗
21. Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P., and Hedman, P. (2022, January 18–24). Mip-nerf 360: Unbounded anti-aliased neural radiance fields. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00539 ↗
22. Chen, S., Zhang, Y., Xu, Y., and Zou, B. (2022). Structure-Aware NeRF without Posed Camera via Epipolar Constraint. arXiv.
23. Wang, Z., Wu, S., Xie, W., Chen, M., and Prisacariu, V.A. (2021). NeRF−: Neural radiance fields without known camera parameters. arXiv.
24. Lin, C.H., Ma, W.C., Torralba, A., and Lucey, S. (2021, January 11–17). Barf: Bundle-adjusting neural radiance fields. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00569 ↗
25. Chng, S.F., Ramasinghe, S., Sherrah, J., and Lucey, S. (2022, January 23–27). Gaussian activated neural radiance fields for high fidelity reconstruction and pose estimation. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-19827-4_16 ↗
26. Meng, Q., Chen, A., Luo, H., Wu, M., Su, H., Xu, L., He, X., and Yu, J. (2021, January 11–17). Gnerf: Gan-based neural radiance field without posed camera. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00629 ↗
27. Hu, T., Liu, S., Chen, Y., Shen, T., and Jia, J. (2022, January 18–24). Efficientnerf efficient neural radiance fields. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01256 ↗
28. Yu, A., Li, R., Tancik, M., Li, H., Ng, R., and Kanazawa, A. (2021, January 11–17). Plenoctrees for real-time rendering of neural radiance fields. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00570 ↗
29. Sun, C., Sun, M., and Chen, H.T. (2022, January 18–24). Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00538 ↗
30. Chen, A., Xu, Z., Geiger, A., Yu, J., and Su, H. (2022, January 23–27). Tensorf: Tensorial radiance fields. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-19824-3_20 ↗
31. Instant neural graphics primitives with a multiresolution hash encoding
32. Li, R., Gao, H., Tancik, M., and Kanazawa, A. (2023). Nerfacc: Efficient sampling accelerates nerfs. arXiv.10.1109/ICCV51070.2023.01699 ↗
33. Liu, S., Lin, S., Lu, J., Saha, S., Supikov, A., and Yip, M. (2023). BAA-NGP: Bundle-Adjusting Accelerated Neural Graphics Primitives. arXiv.
34. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–23). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗