MSACN: A Cloud Extraction Method from Satellite Image Using Multiscale Soft Attention Convolutional Neural Network
Source abstract
Abstract
In satellite remote sensing images, the existence of clouds has an occlusion effect on ground information. Different degrees of clouds make it difficult for existing models to accurately detect clouds in images due to complex scenes. The detection and extraction of clouds is one of the most important problems to be solved in the further analysis and utilization of image information. In this article, we refined a multi-head soft attention convolutional neural network incorporating spatial information modeling (MSACN). During the encoder process, MSACN extracts cloud features through a concurrent dilated residual convolution module. In the part of the decoder, there is an aggregating feature module that uses a soft attention mechanism. It integrates the semantic information with spatial information to obtain the pixel-level semantic segmentation outputs. To assess the applicability of MSACN, we compare our network with Transform-based and other traditional CNN-based methods on the ZY-3 dataset. Experimental outputs including the other two datasets show that MSACN has a better overall performance for cloud extraction tasks, with an overall accuracy of 98.57%, a precision of 97.61%, a recall of 97.37%, and F1-score of 97.48% and an IOU of 95.10%.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Remote-Sensing Image Classification · Advanced Image Fusion Techniques · Remote Sensing in Agriculture
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Science and Engineering, Shenyang Ligong University, Shenyang 110159, China
School of Computer Science and Engineering, Northeastern University, Shenyang 110169, China
Shen Kan Engineering & Technology Corportion, MCC, Shenyang 110169, China
Faculty of Engineering, Gifu University, Gifu 501-1193, Japan
Funding
Liaoning Provincial Department of Education Youth Project · National Natural Science Foundation of China · China Scholarship Council · Liaoning Province Applied Basic Research Program (Youth Special Project) · Shenyang Youth Science and Technology Innovation Talent Support Program
References (37)
Show source-provided references
1. Miroszewski, A., Mielczarek, J., Czelusta, G., Szczepanek, F., Grabowski, B., Le Saux, B., and Nalepa, J. (2023). Detecting Clouds in Multispectral Satellite Images Using Quantum-Kernel Support Vector Machines. arXiv.10.1109/IGARSS52108.2023.10282416 ↗
2. Cloud Detection and Classification Algorithms for Himawari-8 Imager Measurements Based on Deep Learning
3. Ozkan, S., Efendioglu, M., and Demirpolat, C. (2018, January 22–27). Cloud Detection from RGB Color Remote Sensing Images with Deep Pyramid Networks. Proceedings of the IGARSS 2018—2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain.10.1109/IGARSS.2018.8519570 ↗
4. A Machine Learning-Based Cloud Detection Algorithm for the Himawari-8 Spectral Image10.1007/s00376-021-0366-x ↗
5. Massetti, L., Materassi, A., and Sabatini, F. (2023). NSKY-CD: A System for Cloud Detection Based on Night Sky Brightness and Sky Temperature. Remote Sens., 15.10.3390/rs15123063 ↗
6. Zekoll, V., de los Reyes, R., and Richter, R. (2022). A Newly Developed Algorithm for Cloud Shadow Detection—TIP Method. Remote Sens., 14.10.3390/rs14122922 ↗
7. Automatic Cloud and Cloud Shadow Detection in Tropical Areas for PlanetScope Satellite Images10.1016/j.rse.2021.112604 ↗
8. A Coarse-to-Fine Method for Cloud Detection in Remote Sensing Images10.1109/LGRS.2018.2866499 ↗
9. Xiong, Q., Wang, Y., Liu, D., Ye, S., Du, Z., Liu, W., Huang, J., Su, W., Zhu, D., and Yao, X. (2020). A Cloud Detection Approach Based on Hybrid Multispectral Features with Dynamic Thresholds for GF-1 Remote Sensing Images. Remote Sens., 12.10.3390/rs12030450 ↗
10. Cloud Detection Using Sentinel 2 Imageries: A Comparison of XGBoost, RF, SVM, and CNN Algorithms10.1080/10106049.2022.2146211 ↗
11. Energy-Based Cloud Detection in Multispectral Images Based on the SVM Technique10.1080/01431161.2019.1580788 ↗
12. Multi-Spectral Cloud Detection Based on a Multi-Dimensional and Multi-Grained Dense Cascade Forest10.1117/1.JRS.15.028507 ↗
13. Duta, I.C., Liu, L., Zhu, F., and Shao, L. (2020). Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition. arXiv.
14. Cloud detection in RGB color remote sensing images based on improved M-type convolutional network
15. U-Net: Convolutional Networks for Biomedical Image Segmentation
16. Water Body Extraction From Very High-Resolution Remote Sensing Imagery Using Deep U-Net and a Superpixel-Based Conditional Random Field Model10.1109/LGRS.2018.2879492 ↗
17. Wei, S., Zhang, H., Wang, C., Wang, Y., and Xu, L. (2019). Multi-Temporal SAR Data Large-Scale Crop Mapping Based on U-Net Model. Remote Sens., 11.10.3390/rs11010068 ↗
18. Cao, K., and Zhang, X. (2020). An Improved Res-UNet Model for Tree Species Classification Using Airborne High-Resolution Images. Remote Sens., 12.10.3390/rs12071128 ↗
19. CloudU-Net: A Deep Convolutional Neural Network Architecture for Daytime and Nighttime Cloud Images’ Segmentation10.1109/LGRS.2020.3009227 ↗
20. Attention is all you need
21. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
22. Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8–14). CBAM: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
23. Ghaffarian, S., Valente, J., Van Der Voort, M., and Tekinerdogan, B. (2021). Effect of Attention Mechanism in Deep Learning-Based Remote Sensing Image Processing: A Systematic Literature Review. Remote Sens., 13.10.3390/rs13152965 ↗
24. Improving Deep Learning-Based Cloud Detection for Satellite Images With Attention Mechanism
25. Yao, X., Guo, Q., and Li, A. (2021). Light-Weight Cloud Detection Network for Optical Remote Sensing Images with Attention-Based DeeplabV3+ Architecture. Remote Sens., 13.10.3390/rs13183617 ↗
26. Yao, Z., Jia, J., and Qian, Y. (2021). MCNet: Multi-Scale Feature Extraction and Content-Aware Reassembly Cloud Detection Model for Remote Sensing Images. Symmetry, 13.10.3390/sym13010028 ↗
27. Cloud Detection Method Based on Spatial–Spectral Features and Encoder–Decoder Feature Fusion10.1109/TGRS.2023.3329206 ↗
28. Cloud Detection Method Using CNN Based on Cascaded Feature Attention and Channel Attention
29. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
30. Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., and Kainz, B. (2018). Attention U-Net: Learning Where to Look for the Pancreas. arXiv.
31. Cloud Detection Based on Multi-Scale Dilation Convolutional Neural Network for ZY-3 Satellite Remote Sensing Imagery10.3788/AOS201939.0104002 ↗
32. Mohajerani, S., Krammer, T.A., and Saeedi, P. (2018). Cloud Detection Algorithm for Remote Sensing Images Using Fully Convolutional Neural Networks. arXiv.10.1109/MMSP.2018.8547095 ↗
33. Mohajerani, S., and Saeedi, P. (2019). Cloud-Net: An End-to-End Cloud Detection Algorithm for Landsat 8 Imagery. arXiv.10.1109/IGARSS.2019.8898776 ↗
34. Li, Z., Shen, H., Li, H., Xia, G., Gamba, P., and Zhang, L. (2017). Multi-Feature Combined Cloud and Cloud Shadow Detection in GaoFen-1 Wide Field of View Imagery. Remote Sens. Environ.10.1016/j.rse.2017.01.026 ↗
35. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. arXiv.10.1007/978-3-030-01234-2_49 ↗
36. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., and Zhou, Y. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv.
37. Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., and Wang, M. (2021). Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation. arXiv.