Self-Supervised Cloud Classification with Patch Rotation Tasks (SSCC-PR)
Source abstract
Abstract
Solar irradiance, which is closely influenced by cloud cover, significantly affects photovoltaic (PV) power generation efficiency. To improve cloud type recognition without relying on labeled data tasks, this paper proposes a self-supervised cloud classification method based on patch rotation prediction. In the Pre-training stage, unlabeled ground-based cloud images are augmented through blockwise rotation, and high-level semantic representations are learned via a Swin Transformer encoder. In the fine-tuning stage, these representations are adapted to the cloud classification task using labeled data. Experimental results show that our method achieves 96.61% accuracy on the RCCD and 90.18% on the SWIMCAT dataset, outperforming existing supervised and self-supervised baselines by a clear margin. These results demonstrate the effectiveness and robustness of the proposed approach, especially in data-scarce scenarios. This research provides valuable technical support for improving the prediction of solar irradiance and optimizing PV power generation efficiency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Fire Detection and Safety Systems · Data Stream Mining Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Information and Systems Science Institute, Nanjing University of Information Science and Technology, Nanjing 210044, China
Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology, Nanjing University of Information Science and Technology, Nanjing 210044, China
School of Automation, Nanjing University of Information Science and Technology, Nanjing 210044, China
School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China
Funding
National Natural Science Foundation of China under Grant · China Postdoctoral Science Foundation · Postgraduate Research and Practice Innovation Program of Jiangsu Province
References (44)
Show source-provided references
1. Global, seasonal cloud variations from satellite radiance measurements. Part I: Sensitivity of analysis10.1175/1520-0442(1989)002<0419:GSCVFS>2.0.CO;2 ↗
2. Global, seasonal cloud variations from satellite radiance measurements. Part II. Cloud properties and radiative effects10.1175/1520-0442(1990)003<1204:GSCVFS>2.0.CO;2 ↗
3. Remote sensing of cloud properties using ground-based measurements of zenith radiance10.1029/2005JD006843 ↗
4. Determination of three-dimensional cloud structures from high-resolution radiance data10.1029/2005JD006062 ↗
5. Automated ground-based cloud recognition10.1007/s10044-005-0007-5 ↗
6. Feature extraction from whole-sky ground-based images for cloud-type recognition10.1175/2007JTECHA959.1 ↗
7. Cloud classification based on structure features of infrared images10.1175/2010JTECHA1385.1 ↗
8. CloudNet: Ground-based cloud classification with deep convolutional neural network10.1029/2018GL077787 ↗
9. Fang, C., Jia, K., Liu, P., and Zhang, L. (2019, January 18–21). Research on cloud recognition technology based on transfer learning. Proceedings of the 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Lanzhou, China.10.1109/APSIPAASC47483.2019.9023267 ↗
10. Deep convolutional activations-based features for ground-based cloud classification10.1109/LGRS.2017.2681658 ↗
11. Ground-based cloud classification using task-based graph convolutional network10.1029/2020GL087338 ↗
12. Jaiswal, A., Babu, A.R., Zadeh, M.Z., Banerjee, D., and Makedon, F. (2020). A survey on contrastive self-supervised learning. Technologies, 9.10.3390/technologies9010002 ↗
13. Zhai, X., Oliver, A., Kolesnikov, A., and Beyer, L. (2019, January 27–28). S4l: Self-supervised semi-supervised learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.10.1109/ICCV.2019.00156 ↗
14. A survey on self-supervised learning: Algorithms, applications, and future trends10.1109/TPAMI.2024.3415112 ↗
15. Attention is all you need
16. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
17. Review on Convolutional Neural Networks (CNN) in vegetation remote sensing10.1016/j.isprsjprs.2020.12.010 ↗
18. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00986 ↗
19. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
20. Convolutional deep belief networks on cifar-10
21. Cloud adjacency effects on top-of-atmosphere radiance and ocean color data products: A statistical assessment10.1016/j.rse.2015.12.020 ↗
22. Detection and removal of clouds and associated shadows in satellite imagery based on simulated radiance fields10.1029/2018JD029960 ↗
23. Clouds and Earth Radiant Energy System (CERES), a review: Past, present and future10.1016/j.asr.2011.03.009 ↗
24. Dev, S., Lee, Y.H., and Winkler, S. (2015, January 27–30). Categorization of cloud image patches using an improved texton-based approach. Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada.10.1109/ICIP.2015.7350833 ↗
25. Simonyan, K. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
26. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
27. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany. Proceedings, Part III 18.
28. Tan, M., and Le, Q. (2019, January 9–15). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning, PMLR 2019, Long Beach, CA, USA.
29. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
30. Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
31. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗
32. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
33. Gidaris, S., Singh, P., and Komodakis, N. (2018). Unsupervised representation learning by predicting image rotations. arXiv.
34. Bootstrap your own latent-a new approach to self-supervised learning
35. He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020, January 13–19). Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00975 ↗
36. Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020, January 13–18). A simple framework for contrastive learning of visual representations. Proceedings of the International Conference on Machine Learning, PMLR, Virtual Event.
37. Misra, I., and Maaten, L. (2020, January 13–19). Self-supervised learning of pretext-invariant representations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00674 ↗
38. Backpropagation and stochastic gradient descent method10.1016/0925-2312(93)90006-O ↗
39. Cazenave, T., Sentuc, J., and Videau, M. (2021). Cosine annealing, mixnet and swish activation for computer Go. Advances in Computer Games, Springer International Publishing.10.1007/978-3-031-11488-5_5 ↗
40. Liu, Z., Zhou, S., Wang, M., Peng, S., Shen, A., and Zhou, S. (2019, January 21–22). Ground-based visible-light cloud image classification based on a convolutional neural network. Proceedings of the 2019 4th International Conference on Information Systems and Computer Networks (ISCON), Mathura, India.10.1109/ISCON47742.2019.9036273 ↗
41. Fang, H., Han, B., Zhang, S., Zhou, S., Hu, C., and Ye, W.-M. (2024, January 3–8). Data Augmentation for Object Detection via Controllable Diffusion Models. Proceedings of the 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV57701.2024.00129 ↗
42. Improving cloud type classification of ground-based images using region covariance descriptors10.5194/amt-14-737-2021 ↗
43. Sinaga, K.P., and Yang, M.S. (2025). A Globally Collaborative Multi-View k-Means Clustering. Electronics, 14.10.3390/electronics14112129 ↗
44. A proxy learning curve for the Bayes classifier10.1016/j.patcog.2022.109240 ↗