journal article·2022-05-20Open access

An End-to-End Atrous Spatial Pyramid Pooling and Skip-Connections Generative Adversarial Segmentation Network for Building Extraction from High-Resolution Aerial Images

Mingyang Yu, Wenzhuo Zhang, Xiaoxian Chen, Yaohui Liu, Jingge Niu

Applied Sciences· Volume 12· Issue 10

Abstract

Automatic building extraction based on high-resolution aerial imagery is an important challenge with a wide range of practical applications. One of the mainstream methods for extracting buildings from high-resolution images is deep learning because of its excellent deep feature extraction capability. However, existing models suffer from the problems of hollow interiors of some buildings and blurred boundaries. Furthermore, the increase in remote sensing image resolution has also led to rough segmentation results. To address these issues, we propose a generative adversarial segmentation network (ASGASN) for pixel-level extraction of buildings. The segmentation network of this framework adopts an asymmetric encoder–decoder structure. It captures and aggregates multiscale contextual information using the ASPP module and improves the classification and localization accuracy of the network using the global convolutional block. The discriminator network is an adversarial network that correctly discriminates the output of the generator and ground truth maps and computes multiscale L1 loss by fusing multiscale feature mappings. The segmentation network and the discriminator network are trained alternately on the WHU building dataset and the China typical cities building dataset. Experimental results show that the proposed ASGASN can accurately identify different types of buildings and achieve pixel-level high accuracy extraction of buildings. Additionally, compared to available deep learning models, ASGASN also achieved the highest accuracy performance (89.4% and 83.6% IoU on these two datasets, respectively).
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Remote Sensing and LiDAR Applications · Automated Road and Building Extraction · Remote-Sensing Image Classification

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Surveying and Geo-Informatics, Shandong Jianzhu University, Jinan 250101, China

Hebei Key Laboratory of Earthquake Dynamics, Sanhe 065201, China

Funding

National Natural Science Foundation of China · Open Fund Project of Key Laboratory of Earthquake Dynamics in Hebei Province

References (50)

Show source-provided references
1. He, H., Yang, D., Wang, S., Wang, S., and Li, Y. (2019). Road Extraction by Using Atrous Spatial Pyramid Pooling Integrated Encoder-Decoder Network and Structural Similarity Loss. Remote Sens., 11.10.3390/rs11091015 ↗
2. Automatic Building Extraction on High-Resolution Remote Sensing Imagery Using Deep Convolutional Encoder-Decoder with Spatial Pyramid Pooling10.1109/ACCESS.2019.2940527 ↗
3. Boonpook, W., Tan, Y., Ye, Y., Torteeka, P., Torsri, K., and Dong, S. (2018). A Deep Learning Approach on Building Detection from Unmanned Aerial Vehicle-Based Images in Riverbank Monitoring. Sensors, 18.10.3390/s18113921 ↗
4. ARC-Net: An Efficient Network for Building Extraction from High-Resolution Aerial Images10.1109/ACCESS.2020.3015701 ↗
5. Integrating Change Magnitude Maps of Spectrally Enhanced Multi-Features for Land Cover Change Detection10.1080/01431161.2021.1892860 ↗
6. Classification of the Wildland–Urban Interface: A Comparison of Pixel-and Object-Based Classifications Using High-Resolution Aerial Photography10.1016/j.compenvurbsys.2007.10.001 ↗
7. An Efficient Approach for Automatic Rectangular Building Extraction from Very High Resolution Optical Satellite Imagery10.1109/LGRS.2014.2347332 ↗
8. Dalal, N., and Triggs, B. (2005, January 20–25). Histograms of Oriented Gradients for Human Detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA.
9. An Analysis of Co-Occurrence Texture Statistics as a Function of Grey Level Quantization10.5589/m02-004 ↗
10. An Adaptive Change Threshold Selection Method Based on Land Cover Posterior Probability and Spatial Neighborhood Information10.1109/JSTARS.2021.3124491 ↗
11. Kim, T., Lee, T.-Y., Lim, Y.J., and Kim, K.-O. (2005, January 29). The Use of Voting Strategy for Building Extraction from High Resolution Satellite Images. Proceedings of the 2005 IEEE International Geoscience and Remote Sensing Symposium, IGARSS’05, Seoul, Korea.
12. Automatic Building Extraction Based on Multiresolution Segmentation Using Remote Sensing Data10.7163/GPol.0026 ↗
13. Recognition-Driven Two-Dimensional Competing Priors toward Automatic and Accurate Building Detection10.1109/TGRS.2008.2002027 ↗
14. Texture-Based Airport Runway Detection10.1109/LGRS.2012.2210189 ↗
15. Automatic Recognition of Man-Made Objects in High Resolution Optical Remote Sensing Images by SVM Classification of Geometric Image Features10.1016/j.isprsjprs.2007.05.011 ↗
16. One Class Random Forests10.1016/j.patcog.2013.05.022 ↗
17. Guo, J., Pan, Z., Lei, B., and Ding, C. (2017). Automatic Color Correction for Multisource Remote Sensing Images with Wasserstein CNN. Remote Sens., 9.10.3390/rs9050483 ↗
18. Gradient-Based Learning Applied to Document Recognition10.1109/5.726791 ↗
19. Imagenet Classification with Deep Convolutional Neural Networks
20. Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
21. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 8–10). Going Deeper with Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
22. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗
23. Semantic Segmentation of Aerial Images with an Ensemble of CNSS10.5194/isprs-annals-III-3-473-2016 ↗
24. Long, J., Shelhamer, E., and Darrell, T. (2015, January 8–10). Fully Convolutional Networks for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
25. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.10.1007/978-3-319-24574-4_28 ↗
26. Badrinarayanan, V., Kendall, A., and SegNet, R.C. (2015). A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. arXiv.
27. Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2014). Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected Crfs. arXiv.
28. Deeplab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected Crfs10.1109/TPAMI.2017.2699184 ↗
29. Chen, L.-C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking Atrous Convolution for Semantic Image Segmentation. arXiv.
30. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8–14). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_49 ↗
31. Guo, M., Liu, H., Xu, Y., and Huang, Y. (2020). Building Extraction Based on U-Net with an Attention Block and Multiple Losses. Remote Sens., 12.10.3390/rs12091400 ↗
32. Building Footprint Extraction from Unmanned Aerial Vehicle Images via PRU-Net: Application to Change Detection10.1109/JSTARS.2021.3052495 ↗
33. Ye, Z., Fu, Y., Gan, M., Deng, J., Comber, A., and Wang, K. (2019). Building Extraction from Very High Resolution Aerial Imagery Using Joint Attention Deep Neural Network. Remote Sens., 11.10.3390/rs11242970 ↗
34. Generative Adversarial Networks: An Overview10.1109/MSP.2017.2765202 ↗
35. A Novel Framework for Semantic Segmentation with Generative Adversarial Network10.1016/j.jvcir.2018.11.020 ↗
36. Building Footprint Extraction from High Resolution Aerial Images Using Generative Adversarial Network (GAN) Architecture10.1109/ACCESS.2020.3038225 ↗
37. Building Footprint Extraction in Yangon City from Monocular Optical Satellite Image Using Deep Learning
38. Fully Convolutional Networks for Multisource Building Extraction from an Open Aerial and Satellite Imagery Data Set10.1109/TGRS.2018.2858817 ↗
39. A Dataset of Building Instances of Typical Cities in China
40. Learning Multiscale Deep Features for High-Resolution Satellite Image Scene Classification10.1109/TGRS.2017.2743243 ↗
41. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
42. Peng, C., Zhang, X., Yu, G., Luo, G., and Sun, J. (2017, January 21–26). Large Kernel Matters--Improve Semantic Segmentation by Global Convolutional Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.189 ↗
43. Subpixel Mapping of Snow Cover in Forests by Optical Remote Sensing10.1016/S0034-4257(02)00098-6 ↗
44. GIS and Remote Sensing Techniques for the Estimation of Dew Volume in the Republic of Serbia10.1002/met.1930 ↗
45. Zhang, Z., and Wang, Y. (2019). JointNet: A Common Neural Network for Road and Building Extraction. Remote Sens., 11.10.3390/rs11060696 ↗
46. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid Scene Parsing Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
47. Wu, G., Shao, X., Guo, Z., Chen, Q., Yuan, W., Shi, X., Xu, Y., and Shibasaki, R. (2018). Automatic Building Segmentation of Aerial Imagery Using Multi-Constraint Fully Convolutional Networks. Remote Sens., 10.10.3390/rs10030407 ↗
48. BlindNet Backdoor: Attack on Deep Neural Network Using Blind Watermark10.1007/s11042-021-11135-0 ↗
49. Medicalguard: U-Net Model Robust against Adversarially Perturbed Images
50. Data Correction for Enhancing Classification Accuracy by Unknown Deep Neural Network Classifiers