Integrating Convolutional Neural Network and Multiresolution Segmentation for Land Cover and Land Use Mapping Using Satellite Imagery
Source abstract
Abstract
Depletion of natural resources, population growth, urban migration, and expanding drought conditions are some of the reasons why environmental monitoring programs are required and regularly produced and updated. Additionally, the usage of artificial intelligence in the geospatial field of Earth observation (EO) and regional land monitoring missions is a challenging issue. In this study, land cover and land use mapping was performed using the proposed CNN–MRS model. The CNN–MRS model consisted of two main steps: CNN-based land cover classification and enhancing the classification with spatial filter and multiresolution segmentation (MRS). Different band numbers of Sentinel-2A imagery and multiple patch sizes (32 × 32, 64 × 64, and 128 × 128 pixels) were used in the first experiment. The algorithms were evaluated in terms of overall accuracy, precision, recall, F1-score, and kappa coefficient. The highest overall accuracy was obtained with the proposed approach as 97.31% in Istanbul test site area and 98.44% in Kocaeli test site area. The accuracies revealed the efficiency of the CNN–MRS model for land cover map production in large areas. The McNemar test measured the significance of the models used. In the second experiment, with the Zurich Summer dataset, the overall accuracy of the proposed approach was obtained as 92.03%. The results are compared quantitatively with state-of-the-art CNN model results and related works.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Remote-Sensing Image Classification · Remote Sensing in Agriculture · Remote Sensing and Land Use
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Geomatics Engineering, Istanbul Technical University, Maslak, Istanbul 34469, Turkey
Funding
Bilimsel Araştırma Projeleri Birimi, İstanbul Teknik Üniversitesi
References (63)
Show source-provided references
1. Evaluation of the vegetated urban canopy model (VUCM) and its impacts on urban boundary layer simulation10.1007/s13143-011-0005-z ↗
2. (2020, December 25). Copernicus-CORINE Land Cover. Available online: https://land.copernicus.eu/pan-european/corine-land-cover.
3. Development of a land-cover characteristics database for the conterminous
4. Mapping of land use classes within the CORINE land cover map of Great Britain10.1179/caj.2002.39.1.5 ↗
5. Wetland classification for Swedish CORINE Land Cover adopting a semi-automatic interactive approach10.5589/m02-011 ↗
6. Vali, A., Comai, S., and Matteucci, M. (2020). Deep learning for land use and land cover classification based on hyperspectral and multi-spectral earth observation data: A review. Remote Sens., 12.10.3390/rs12152495 ↗
7. Pan-European forest/non-forest mapping with Landsat ETM+ and CORINE Land Cover 2000 data10.1016/j.isprsjprs.2008.09.004 ↗
8. Heryadi, Y., and Miranda, E. (2019). Land Cover Classification Based on Sentinel-2 Satellite Imagery Using Convolutional Neural Network Model: A Case Study in Semarang Area, Indonesia. Asian Conference on Intelligent Information and Database Systems, Springer.10.1007/978-3-030-14132-5_15 ↗
9. Qin, Y., Wu, Y., Li, B., Gao, S., Liu, M., and Zhan, Y. (2019). Semantic Segmentation of Building Roof in Dense Urban Environment with Deep Convolutional Neural Network: A Case Study Using GF2 VHR Imagery in China. Sensors, 19.10.3390/s19051164 ↗
10. Semantic segmentation of slums in satellite images using transfer learning on fully convolutional neural networks10.1016/j.isprsjprs.2019.02.006 ↗
11. Semantic Segmentation of Brazilian Savanna Vegetation Using High Spatial Resolution Satellite Data and U-Net10.5194/isprs-annals-V-3-2020-505-2020 ↗
12. Yao, X., Yang, H., Wu, Y., Wu, P., Wang, B., Zhou, X., and Wang, S. (2019). Land Use Classification of the Deep Convolutional Neural Network Method Reducing the Loss of Spatial Features. Sensors, 19.10.3390/s19122792 ↗
13. Demir, I., Koperski, K., Lindenbaum, D., Pang, G., Huang, J., Basu, S., Hughes, F., Tuia, D., and Raskar, R. (2018, January 18–22). DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.10.1109/CVPRW.2018.00031 ↗
14. Sumbul, G., Charfuelan, M., Demir, B., and Markl, V. (August, January 28). Bigearthnet: A Large-Scale Benchmark Archive for Remote Sensing Image Understanding. Proceedings of the IGARSS 2019—2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan.10.1109/IGARSS.2019.8900532 ↗
15. Wang, S., Chen, W., Xie, S.M., Azzari, G., and Lobell, D.B. (2020). Weakly Supervised Deep Learning for Segmentation of Remote Sensing Imagery. Remote Sens., 12.10.3390/rs12020207 ↗
16. Audebert, N., Le Saux, B., and Lefèvre, S. (2016). Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks. Asian Conference on Computer Vision, Springer.
17. Algorithms for semantic segmentation of multi-spectral remote sensing imagery using deep learning10.1016/j.isprsjprs.2018.04.014 ↗
18. Change detection using deep learning approach with object-based image analysis10.1016/j.rse.2021.112308 ↗
19. Transferable instance segmentation of dwellings in a refugee camp—Integrating CNN and OBIA10.1080/22797254.2020.1759456 ↗
20. Papadomanolaki, M., Vakalopoulou, M., and Karantzalos, K. (2019). A novel object-based deep learning framework for semantic seg-mentation of very high-resolution remote sensing data: Comparison with convolutional and fully convolutional networks. Remote Sens., 11.10.3390/rs11060684 ↗
21. Exploring multiscale object-based convolutional neural network (multi-OCNN) for remote sensing image classification at high spatial resolution10.1016/j.isprsjprs.2020.08.004 ↗
22. Comparing fully convolutional networks, random forest, support vector machine, and patch-based deep convolutional neural networks for object-based wetland mapping using images from small unmanned aircraft system10.1080/15481603.2018.1426091 ↗
23. Li, E., Samat, A., Liu, W., Lin, C., and Bai, X. (2019). High-Resolution Imagery Classification Based on Different Levels of Information. Remote Sens., 11.10.3390/rs11242916 ↗
24. Decision Fusion with Multiple Spatial Supports by Conditional Random Fields10.1109/TGRS.2018.2797316 ↗
25. Volpi, M., and Ferrari, V. (2015, January 7–12). Semantic segmentation of urban scenes by learning local class interactions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA.10.1109/CVPRW.2015.7301377 ↗
26. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
27. Segnet: A deep convolutional encoder-decoder architecture for image segmen-tation10.1109/TPAMI.2016.2644615 ↗
28. Abdollahi, A., Pradhan, B., and Alamri, A.M. (2020). An ensemble architecture of deep convolutional Segnet and Unet networks for building semantic segmentation from high-resolution aerial images. Geocarto Int., 1–16.10.1080/10106049.2020.1856199 ↗
29. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
30. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Cham, Switzerland.10.1007/978-3-319-24574-4_28 ↗
31. Strobl, J., Blaschke, T., and Griesebner, G. (2000, January 30). Multiresolution segmentation: An optimization approach for high quality multi-scale image segmen-tation. Proceedings of the Angewandte Geographische Informations-Verarbeitung XII, Karlsruhe, Germany.
32. Segmentation for Object-Based Image Analysis (OBIA): A review of algorithms and challenges from remote sensing perspective10.1016/j.isprsjprs.2019.02.009 ↗
33. A multi-resolution remotely sensed image segmentation method combining rainfalling watershed algorithm and fast region merging
34. Optimization in multi-scale segmentation of high-resolution satellite images for artificial feature recognition10.1080/01431160701241746 ↗
35. Roni, R., and Jia, P. (2020). An Optimal Population Modeling Approach Using Geographically Weighted Regression Based on High-Resolution Remote Sensing Data: A Case Study in Dhaka City, Bangladesh. Remote Sens., 12.10.3390/rs12071184 ↗
36. Lang, S., Hay, G.J., Baraldi, A., Tiede, D., and Blaschke, T. (2019). Geobia Achievements and Spatial Opportunities in the Era of Big Earth Observation Data. ISPRS Int. J. Geo-Inf., 8.10.3390/ijgi8110474 ↗
37. Tetteh, G., Gocht, A., Schwieder, M., Erasmi, S., and Conrad, C. (2020). Unsupervised Parameterization for Optimal Segmentation of Agricultural Parcels from Satellite Images in Different Agricultural Landscapes. Remote Sens., 12.10.3390/rs12183096 ↗
38. Segmentation of large-scale remotely sensed images on a Spark platform: A strategy for handling massive image tiles with the MapReduce model10.1016/j.isprsjprs.2020.02.012 ↗
39. Ding, H., Liu, K., Chen, X., Xiong, L., Tang, G., Qiu, F., and Strobl, J. (2020). Optimized Segmentation Based on the Weighted Aggregation Method for Loess Bank Gully Mapping. Remote Sens., 12.10.3390/rs12050793 ↗
40. An experimental comparison of multi-resolution segmentation, SLIC and K-means clustering for object-based classification of VHR imagery10.1080/01431161.2018.1506592 ↗
41. Rouse, J.W., Haas, R.H., Schell, J.A., and Deering, D.W. (1974). Monitoring Vegetation Systems in the Great Plains with ERTS, NASA. NASA Special Publication, 351.
42. Distinguishing vegetation from soil background information
43. Remote mapping of standing crop biomass for estimation of the productivity of the shortgrass prairie
44. Red and photographic infrared linear combinations for monitoring vegetation10.1016/0034-4257(79)90013-0 ↗
45. Alpaydın, E. (2010). Introduction to Machine Learning, MIT Press. [2nd ed.].
46. A review of assessing the accuracy of classifications of remotely sensed data10.1016/0034-4257(91)90048-B ↗
47. A Coefficient of Agreement for Nominal Scales10.1177/001316446002000104 ↗
48. Chatziantoniou, A., Psomiadis, E., and Petropoulos, G.P. (2017). Co-Orbital Sentinel 1 and 2 for LULC mapping with emphasis on wet-lands in a mediterranean setting based on machine learning. Remote Sens., 9.10.3390/rs9121259 ↗
49. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation
50. Note on the sampling error of the difference between correlated proportions or percentages10.1007/BF02295996 ↗
51. Thematic map comparison: Evaluating the statistical significance of differences in classification accuracy10.14358/PERS.70.5.627 ↗
52. Li, H., Wang, C., Zhong, C., Zhang, Z., and Liu, Q. (2017). Mapping Typical Urban LULC from Landsat Imagery without Training Samples or Self-Defined Parameters. Remote Sens., 9.10.3390/rs9070700 ↗
53. Hütt, C., Koppe, W., Miao, Y., and Bareth, G. (2016). Best Accuracy Land Use/Land Cover (LULC) Classification to Derive Crop Types Using Multitemporal, Multisensor, and Multi-Polarization SAR Satellite Images. Remote Sens., 8.10.3390/rs8080684 ↗
54. Quan, Y., Tong, Y., Feng, W., Dauphin, G., Huang, W., and Xing, M. (2020). A Novel Image Fusion Method of Multi-Spectral and SAR Images for Land Cover Classification. Remote Sens., 12.10.3390/rs12223801 ↗
55. Accuracy Assessment Measures for Object-based Image Segmentation Goodness10.14358/PERS.76.3.289 ↗
56. Kavzoglu, T., and Tonbul, H. (2017, January 19–22). A comparative study of segmentation quality for multi-resolution segmentation and watershed transform. Proceedings of the 2017 8th International Conference on Recent Advances in Space Technologies (RAST), Istanbul, Turkey.10.1109/RAST.2017.8002984 ↗
57. Existential uncertainty of spatial objects segmented from satellite sensor imagery10.1109/TGRS.2002.805072 ↗
58. Location similarity of regions10.1016/S0924-2716(00)00019-8 ↗
59. Note on the “correction for continuity” in testing the significance of the difference between correlated proportions10.1007/BF02289261 ↗
60. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
61. Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv.
62. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8–14). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_49 ↗
63. Incorporating DeepLabv3+ and object-based image analysis for semantic segmentation of very high resolution remote sensing images