A Multiscale Multi-Feature Deep Learning Model for Airborne Point-Cloud Semantic Segmentation
Source abstract
Abstract
In point-cloud scenes, semantic segmentation is the basis for achieving an understanding of a 3D scene. The disorderly and irregular nature of 3D point clouds makes it impossible for traditional convolutional neural networks to be applied directly, and most deep learning point-cloud models often suffer from an inadequate utilization of spatial information and of other related point-cloud features. Therefore, to facilitate the capture of spatial point neighborhood information and obtain better performance in point-cloud analysis for point-cloud semantic segmentation, a multiscale, multi-feature PointNet (MSMF-PointNet) deep learning point-cloud model is proposed in this paper. MSMF-PointNet is based on the classical point-cloud model PointNet, and two small feature-extraction networks called Mini-PointNets are added to operate in parallel with the modified PointNet; these additional networks extract multiscale, multi-neighborhood features for classification. In this paper, we use the spherical neighborhood method to obtain the local neighborhood features of the point cloud, and then we adjust the radius of the spherical neighborhood to obtain the multiscale point-cloud features. The obtained multiscale neighborhood feature point set is used as the input of the network. In this paper, a cross-sectional comparison analysis is conducted on the Vaihingen urban test dataset from the single-scale and single-feature perspectives.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Remote Sensing and LiDAR Applications · 3D Shape Modeling and Analysis · 3D Surveying and Cultural Heritage
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Surveying and Geo-Informatics, North China University of Water Resources and Electric Power, Zhengzhou 450045, China
School of Computer Science, Hubei University of Technology, Wuhan 430068, China
Funding
National Natural Science Foundation of China · Henan Province young talent support project
References (34)
Show source-provided references
1. Understanding an Urbanizing Planet: Strategic Directions for Remote Sensing10.1016/j.rse.2019.04.020 ↗
2. LiDAR Point Clouds to 3-D Urban Models: A Review10.1109/JSTARS.2017.2781132 ↗
3. Airborne LiDAR for DEM generation: Some critical issues10.1177/0309133308089496 ↗
4. Chen, J., Zhao, Y., Meng, C., and Liu, Y. (2022). Multi-Feature Aggregation for Semantic Segmentation of an Urban Scene Point Cloud. Remote Sen., 14.10.3390/rs14205134 ↗
5. BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation10.1007/s11263-021-01515-2 ↗
6. Xu, C., Wu, B., Wang, Z., Zhan, W., Vajda, P., Keutzer, K., and Tomizuka, M. (2020). SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation. arXiv.10.1007/978-3-030-58604-1_1 ↗
7. Wu, B., Zhou, X., Zhao, S., Yue, X., and Keutzer, K. (2019, January 20–24). SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8793495 ↗
8. 3D-MiniNet: Learning a 2D Representation from Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentation10.1109/LRA.2020.3007440 ↗
9. MiniNet: An Efficient Semantic Segmentation ConvNet for Real-Time Robotic Applications10.1109/TRO.2020.2974099 ↗
10. C2FNet: A Coarse-to-Fine Network for Multi-View 3D Point Cloud Generation10.1109/TIP.2022.3203213 ↗
11. Alkadri, M.F., Luca, F.D., Turrin, M., and Sariyildiz, S. (2020). A Computational Workflow for Generating A Voxel-Based Design Approach Based on Subtractive Shading Envelopes and Attribute Information of Point Cloud Data. Remote Sens., 12.10.3390/rs12162561 ↗
12. Zhao, L., Xu, S., Liu, L., Ming, D., and Tao, W. (2022). SVASeg: Sparse Voxel-Based Attention for 3D LiDAR Point Cloud Semantic Segmentation. Remote Sens., 14.10.3390/rs14184471 ↗
13. Charles, R.Q., Su, H., Kaichun, M., and Guibas, L.J. (2017, January 21–26). PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.16 ↗
14. Adaptive deep learning-based neighborhood search method for point cloud10.1038/s41598-022-06200-z ↗
15. Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., and Guibas, L. (November, January 27). KPConv: Flexible and Deformable Convolution for Point Clouds. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00651 ↗
16. ConvPoint: Continuous convolutions for point cloud processing10.1016/j.cag.2020.02.005 ↗
17. 3D Point Cloud Classification and Segmentation Model Based on Graph Convolutional Network10.3788/LOP57.181019 ↗
18. Zeng, Z., Xu, Y., Xie, Z., Wan, J., Wu, W., and Dai, W. (2022). RG-GCN: A Random Graph Based on Graph Convolution Network for Point Cloud Semantic Segmentation. Remote Sens., 14.10.3390/rs14164055 ↗
19. Huang, Q., Wang, W., and Neumann, U. (2018, January 18–23). Recurrent Slice Networks for 3D Segmentation of Point Clouds. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00278 ↗
20. Qi, C.R., Li, Y., Hao, S., and Guibas, L.J. (2017, January 4–9). PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
21. Point cloud classification network based on self-attention mechanism10.1016/j.compeleceng.2022.108451 ↗
22. Yang, J., Zhang, X., and Huang, Y. (2021). Graph Attention Feature Fusion Network for ALS Point Cloud Classification. Sensors, 21.10.3390/s21186193 ↗
23. Luo, B., Yang, J., Song, S., Shi, S., Gong, W., Wang, A., and Du, L. (2022). Target Classification of Similar Spatial Characteristics in Complex Urban Areas by Using Multispectral LiDAR. Remote Sens., 14.10.3390/rs14010238 ↗
24. AGF-Net: Attentive geometric feature pyramid network for land cover classification using airborne multispectral LiDAR data
25. Semantic Segmentation of Indoor 3d Point Clouds by Joint Optimization of Geometric Features and Neural Networks10.5194/isprs-annals-X-4-W2-2022-305-2022 ↗
26. Meng, F., Wang, X., Shao, F., Wang, D., and Hua, X. (2019). Energy-Efficient Gabor Kernels in Neural Networks with Genetic Algorithm Training Method. Electronics, 8.10.3390/electronics8010105 ↗
27. Lai, X., Yang, J., Li, Y., and Wang, M. (2019). A Building Extraction Approach Based on the Fusion of LiDAR Point Cloud and Elevation Map Texture Features. Remote Sens., 11.10.3390/rs11141636 ↗
28. Classification Algorithm for Laser Point Clouds of High-steep Slopes Based on Multi-scale Dimensionality Features and SVM
29. Kingma, D.P., and Ba, J. (2015). Adam: A Method for Stochastic Optimization. arXiv.
30. Shang, W., Sohn, K., Almeida, D., and Lee, H. (2016, January 19–24). Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA.
31. A Fully Convolutional Network for Semantic Labeling of 3D Point Clouds10.1016/j.isprsjprs.2018.03.018 ↗
32. A survey on deep learning techniques for image and video semantic segmentation10.1016/j.asoc.2018.05.018 ↗
33. Yang, Z., Tan, B., Pei, H., and Jiang, W. (2018). Segmentation and Multi-Scale Convolutional Neural Network-Based Classification of Airborne Laser Scanner Data. Sensors, 18.10.3390/s18103347 ↗
34. Classifying airborne LiDAR point clouds via deep features learned by a multi-scale convolutional neural network10.1080/13658816.2018.1431840 ↗