Vegetation Removal on 3D Point Cloud Reconstruction of Cut-Slopes Using U-Net
Source abstract
Abstract
The 3D point cloud reconstruction from photos taken by an unmanned aerial vehicle (UAV) is a promising tool for monitoring and managing risks of cut-slopes. However, surface changes on cut-slopes are likely to be hidden by seasonal vegetation variations on the cut-slopes. This paper proposes a vegetation removal method for 3D reconstructed point clouds using (1) a 2D image segmentation deep learning model and (2) projection matrices available from photogrammetry. For a given point cloud, each 3D point of it is reprojected into the image coordinates by the projection matrices to determine if it belongs to vegetation or not using the 2D image segmentation model. The 3D points belonging to vegetation in the 2D images are deleted from the point cloud. The effort to build a 2D image segmentation model was significantly reduced by using U-Net with the dataset prepared by the colour index method complemented by manual trimming. The proposed method was applied to a cut-slope in Doam Dam in South Korea, and showed that vegetation from the two point clouds of the cut-slope at winter and summer was removed successfully. The M3C2 distance between the two vegetation-removed point clouds showed a feasibility of the proposed method as a tool to reveal actual change of cut-slopes without the effect of vegetation.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Remote Sensing and LiDAR Applications · 3D Surveying and Cultural Heritage · Remote Sensing in Agriculture
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter EX4 4PY, UK
Funding
Korea Agency for Infrastructure Technology Advancement
References (35)
Show source-provided references
1. 3D reconstruction of existing concrete bridges using optical methods10.1080/15732479.2019.1594315 ↗
2. Tang, S., Zhang, Y., Li, Y., Yuan, Z., Wang, Y., Zhang, X., Li, X., Zhang, Y., Guo, R., and Wang, W. (2019). Fast and automatic reconstruction of semantically rich 3D indoor maps from low-quality RGB-D sequences. Sensors, 19.10.3390/s19030533 ↗
3. Applications of photogrammetry to structural assessment10.1111/j.1747-1567.2011.00731.x ↗
4. Unmanned aerial vehicle inspection of the Placer River Trail Bridge through image-based 3D modelling10.1080/15732479.2017.1330891 ↗
5. Congress, S.S.C., and Puppala, A.J. (2019). Novel Methodology of Using Aerial Close Range Photogrammetry Technology for Monitoring the Pavement Construction Projects, American Society of Civil Engineers.10.1061/9780784482476.014 ↗
6. Image-based 3D reconstruction using traditional and UAV datasets for analysis of road pavement distress10.1016/j.autcon.2018.10.010 ↗
7. Integrating 3D laser scanning and photogrammetry for progress measurement of construction work10.1016/j.autcon.2008.05.006 ↗
8. Towards an automated photogrammetry-based approach for monitoring and controlling construction site activities10.1016/j.compind.2018.03.012 ↗
9. Deep convolutional neural network training enrichment using multi-view object-based analysis of Unmanned Aerial systems imagery for wetlands classification10.1016/j.isprsjprs.2018.03.006 ↗
10. Determining ground elevations covered by vegetation on construction sites using drone-based orthoimage and convolutional neural network10.1061/(ASCE)CP.1943-5487.0000930 ↗
11. Detection and geometric characterization of rock mass discontinuities using a 3D high-resolution digital outcrop model generated from RPAS imagery—Ormea rock slope, Italy10.1016/j.enggeo.2019.02.028 ↗
12. Mesas-Carrascosa, F.J., de Castro, A.I., Torres-Sánchez, J., Triviño-Tarradas, P., Jiménez-Brenes, F.M., García-Ferrer, A., and López-Granados, F. (2020). Classification of 3D Point Clouds Using Color Vegetation Indices for Precision Viticulture and Digitizing Applications. Remote Sens., 12.10.3390/rs12020317 ↗
13. Bassine, F.Z., Errami, A., and Khaldoun, M. (2019, January 10–14). Vegetation Recognition Based on UAV Image Color Index. Proceedings of the 2019 IEEE International Conference on Environment and Electrical Engineering and 2019 IEEE Industrial and Commercial Power Systems Europe (EEEIC/I&CPS Europe), Genova, Italy.10.1109/EEEIC.2019.8783830 ↗
14. Zhang, W., Qi, J., Wan, P., Wang, H., Xie, D., Wang, X., and Yan, G. (2016). An Easy-to-Use Airborne LiDAR Data Filtering Method Based on Cloth Simulation. Remote Sens., 8.10.3390/rs8060501 ↗
15. Štroner, M., Urban, R., Lidmila, M., Kolář, V., and Křemen, T. (2021). Vegetation Filtering of a Steep Rugged Terrain: The Performance of Standard Algorithms and a Newly Proposed Workflow on an Example of a Railway Ledge. Remote Sens., 13.10.3390/rs13153050 ↗
16. Classification of airborne 3D point clouds regarding separation of vegetation in complex environments10.1364/AO.422973 ↗
17. Weidner, L.M. (2021). Generalized Machine-Learning-Based Point Cloud Classification for Natural and Cut Slopes. [Ph.D. Thesis, Colorado School of Mines].
18. Pinto, M.F., Melo, A.G., Honório, L.M., Marcato, A.L.M., Conceição, A.G.S., and Timotheo, A.O. (2020). Deep Learning Applied to Vegetation Identification and Removal Using Multidimensional Aerial Data. Sensors, 20.10.3390/s20216187 ↗
19. Anders, N., Valente, J., Masselink, R., and Keesstra, S. (2019). Comparing filtering techniques for removing vegetation from UAV-based photogrammetric point clouds. Drones, 3.10.3390/drones3030061 ↗
20. Becker, C., Häni, N., Rosinskaya, E., d’Angelo, E., and Strecha, C. (2017). Classification of Aerial Photogrammetric 3D Point Clouds. arXiv.10.5194/isprs-annals-IV-1-W1-3-2017 ↗
21. Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., and Xiao, J. (2015). 3D ShapeNets: A Deep Representation for Volumetric Shapes. arXiv.
22. Qi, C.R., Su, H., Mo, K., and Guibas, L.J. (2017). PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. arXiv.
23. Hackel, T., Savinov, N., Ladicky, L., Wegner, J.D., Schindler, K., and Pollefeys, M. (2017). Semantic3D.Net: A New Large-Scale point cloud Classification Benchmark. arXiv.10.5194/isprs-annals-IV-1-W1-91-2017 ↗
24. Van Ginneken, B., Heimann, T., and Styner, M. (2007, January 29). 3D segmentation in the clinic: A grand challenge. Proceedings of the MICCAI Workshop on 3D Segmentation in the Clinic: A Grand Challenge, Brisbane, Australia.
25. Schönberger, J.L., and Frahm, J.M. (2016, January 27–30). Structure-from-Motion Revisited. Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.445 ↗
26. Vegetation segmentation robust to illumination variations based on clustering and morphology modelling10.1016/j.biosystemseng.2014.06.015 ↗
27. Automatic segmentation of relevant textures in agricultural images10.1016/j.compag.2010.09.013 ↗
28. Greenness identification based on HSV decision tree
29. Hassanein, M., Lari, Z., and El-Sheimy, N. (2018). A new vegetation segmentation approach for cropped fields based on threshold detection from hue histograms. Sensors, 18.10.3390/s18041253 ↗
30. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
31. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
32. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18–23). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
33. (2021, December 24). CloudCompare. Available online: https://www.cloudcompare.org/.
34. (2021, December 24). Pix 4D Pix4Dmapper. Available online: https://www.pix4d.com/.
35. 3-D uncertainty-based topographic change detection with structure-from-motion photogrammetry: Precision maps for ground control and directly georeferenced surveys10.1002/esp.4125 ↗