Fast 3D Reconstruction of UAV Images Based on Neural Radiance Field
Source abstract
Abstract
Traditional methods for 3D reconstruction of unmanned aerial vehicle (UAV) images often rely on classical multi-view 3D reconstruction techniques. This classical approach involves a sequential process encompassing feature extraction, matching, depth fusion, point cloud integration, and mesh creation. However, these steps, particularly those that feature extraction and matching, are intricate and time-consuming. Furthermore, as the number of steps increases, a corresponding amplification of cumulative error occurs, leading to its continual augmentation. Additionally, these methods typically utilize explicit representation, which can result in issues such as model discontinuity and missing data during the reconstruction process. To effectively address the challenges associated with heightened temporal expenditures, the absence of key elements, and the fragmented models inherent in three-dimensional reconstruction using Unmanned Aerial Vehicle (UAV) imagery, an alternative approach is introduced—the neural radiance field. This novel method leverages neural networks to intricately fit spatial information within the scene, thereby streamlining the reconstruction steps and rectifying model deficiencies. The neural radiance field method employs a fully connected neural network to meticulously model object surfaces and directly generate the 3D object model. This methodology simplifies the intricacies found in conventional 3D reconstruction processes. Implicitly encapsulating scene characteristics, the neural radiance field allows for iterative refinement of neural network parameters via the utilization of volume rendering techniques. Experimental results substantiate the efficacy of this approach, demonstrating its ability to complete scene reconstruction within a mere 5 min timeframe, thereby reducing reconstruction time by 90% while markedly enhancing reconstruction quality.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: 3D Surveying and Cultural Heritage · Advanced Vision and Imaging · Robotics and Sensor-Based Localization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Geomatics Science and Technology, Nanjing Tech University, Nanjing 211816, China
References (22)
Show source-provided references
1. Hartley, R., and Zisserman, A. (2003). Multiple View Geometry in Computer Vision, Cambridge University Press.10.1017/CBO9780511811685 ↗
2. Gipuma: Massively parallel multi-view stereo reconstruction
3. Agrawal, M., and Davis, L.S. (2001, January 8–14). A probabilistic framework for surface reconstruction from multiple images. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, Kauai, HI, USA.
4. PatchMatch: A randomized correspondence algorithm for structural image editing10.1145/1531326.1531330 ↗
5. De Bonet, J.S., and Viola, P. (1999, January 20–27). Poxels: Probabilistic voxelized volume reconstruction. Proceedings of the International Conference on Computer Vision (ICCV), Corfu, Greece.10.1109/ICCV.1999.791251 ↗
6. Broadhurst, A., Drummond, T.W., and Cipolla, R. (2001, January 7–14). A probabilistic framework for space carving. Proceedings of the Eighth IEEE International Conference on Computer Vision, ICCV 2001, Vancouver, BC, Canada.
7. A theory of shape by space carving10.1023/A:1008191222954 ↗
8. Cooperative Computation of Stereo Disparity: A cooperative algorithm is derived for extracting disparity information from stereo image pairs10.1126/science.968482 ↗
9. Paschalidou, D., Ulusoy, O., Schmitt, C., Van Gool, L., and Geiger, A. (2018, January 18–22). Raynet: Learning volumetric 3d reconstruction with ray potentials. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00410 ↗
10. Accurate, dense, and robust multiview stereopsis10.1109/TPAMI.2009.161 ↗
11. Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J., Yang, R., Nistér, D., and Pollefeys, M. (2007, January 14–21). Real-time visibility-based fusion of depth maps. Proceedings of the 2007 IEEE 11th International Conference on Computer Vision, Rio De Janeiro, Brazil.10.1109/ICCV.2007.4408984 ↗
12. Zach, C., Pock, T., and Bischof, H. (2007, January 14–21). A globally optimal algorithm for robust tv-l 1 range image integration. Proceedings of the 2007 IEEE 11th International Conference on Computer Vision, Rio De Janeiro, Brazil.10.1109/ICCV.2007.4408983 ↗
13. Screened poisson surface reconstruction10.1145/2487228.2487237 ↗
14. Photorealistic scene reconstruction by voxel coloring10.1023/A:1008176507526 ↗
15. Tulsiani, S., Zhou, T., Efros, A.A., and Malik, J. (2017, January 21–26). Multi-view supervision for single-view reconstruction via differentiable ray consistency. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.30 ↗
16. Nerf: Representing scenes as neural radiance fields for view synthesis10.1145/3503250 ↗
17. Jiang, S., Jiang, W., Huang, W., and Yang, L. (2017). UAV-based oblique photogrammetry for outdoor data acquisition and offsite visual inspection of transmission line. Remote Sens., 9.10.3390/rs9030278 ↗
18. Tridimensional reconstruction applied to cultural heritage with the use of camera-equipped UAV and terrestrial laser scanner10.3390/rs61110413 ↗
19. Schonberger, J.L., and Frahm, J. (2016, January 27–30). Structure-from-motion revisited. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.445 ↗
20. Yao, Y., Luo, Z., Li, S., Fang, T., and Quan, L. (2018, January 8–14). Mvsnet: Depth inference for unstructured multi-view stereo. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01237-3_47 ↗
21. Fourier features let networks learn high frequency functions in low dimensional domains
22. Instant neural graphics primitives with a multiresolution hash encoding