Image Deraining Algorithm Based on Multi-Scale Features
Source abstract
Abstract
In target detection, tracking, and recognition tasks, high-quality images can achieve better results. However, in actual scenarios, the visual effects and data quality of images are greatly reduced due to the influence of environmental factors, which affect subsequent detection, recognition, and other tasks. Therefore, this paper proposes an image rain removal algorithm based on multi-scale features, which can effectively remove rain streaks. First of all, this paper proposes a deraining algorithm that combines spatial information to improve the network’s generalization ability on real images, aiming at the problem of synthetic datasets used by previous deraining algorithms. Then, by proposing a multi-scale rain removal algorithm, it improves the feature extraction capabilities of existing deraining algorithms. Before extracting deep rain features, a preliminary fusion of multi-scale shallow features can be performed, which can show better performance in images of different sizes. In addition, a spatial attention module and channel are introduced. The attention module increases the ability to extract rain information at each scale; the resulting multi-scale feature image rain removal algorithm is called MFD. Finally, the rain removal algorithm is validated on the rain removal dataset, and the proposed method can effectively remove rain patterns, provide strong performance improvement on several datasets in the image rain removal task, and provide high-quality images for subsequent detection and recognition tasks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Image Enhancement Techniques · Advanced Image Fusion Techniques · Image and Signal Denoising Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronics and Control Engineering, Chang’an University, Xi’an 710064, China
School of Energy and Electrical Engineering, Chang’an University, Xi’an 710064, China
Funding
National Natural Science Foundation of China
References (43)
Show source-provided references
1. Xiao, J., Zhou, M., Fu, X., Liu, A., and Zha, Z.-J. (2021, January 10–17). Improving De-Raining Generalization via Neural Reorganization. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00494 ↗
2. Liu, S., Huang, D., and Wang, Y. (2018, January 8–14). Receptive Field Block Net for Accurate and Fast Object Detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01252-6_24 ↗
3. Garg, K., and Nayar, S.K. (2014, January 23–28). Detection and Removal of Rain from Videos. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.
4. Vision and Rain10.1007/s11263-006-0028-6 ↗
5. Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition10.1109/TIP.2011.2179057 ↗
6. Luo, Y., Xu, Y., and Ji, H. (2015, January 7–13). Removing Rain from a Single Image via Discriminative Sparse Coding. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.388 ↗
7. Yasarla, R., and Patel, V.M. (2019, January 15–20). Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00860 ↗
8. Zhou, M., Xiao, J., Chang, Y., Fu, X., Liu, A., Pan, J., and Zha, Z.J. (2021, January 20–25). Image De-raining via Continual Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00487 ↗
9. Nanba, Y., Miyata, H., and Han, X. (2022, January 18–24). Dual Heterogeneous Complementary Networks for Single Image Deraining. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPRW56347.2022.00072 ↗
10. Li, X., Wu, J., Lin, Z., Liu, H., and Zha, H. (2018, January 8–14). Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_16 ↗
11. Lightweight Pyramid Networks for Image Deraining10.1109/TNNLS.2019.2926481 ↗
12. Wei, Y., Zhang, Z., Wang, Y., Zhang, H., Zhao, M., Xu, M., and Wang, M. (2021, January 5–9). Semi-Deraingan: A New Semi-Supervised Single Image Deraining. Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China.10.1109/ICME51207.2021.9428285 ↗
13. DerainCycleGAN: Rain Attentive CycleGAN for Single Image Deraining and Rainmaking10.1109/TIP.2021.3074804 ↗
14. Chen, X., Pan, J., Jiang, K., Li, Y., Huang, Y., Kong, C., Dai, L., and Fan, Z. (2022, January 18–24). Unpaired Deep Image Deraining Using Dual Contrastive Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00206 ↗
15. Non-local Channel Aggregation Network for Single Image Rain Removal10.1016/j.neucom.2021.10.052 ↗
16. Ye, Y., Yu, C., Chang, Y., Zhu, L., Zhao, X.L., Yan, L., and Tian, Y. (2022, January 18–24). Unsupervised Deraining: Where Contrastive Learning Meets Self-Similarity. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00573 ↗
17. Yu, C., Chang, Y., Li, Y., Zhao, X., and Yan, L. (2021, January 20–24). Unsupervised Image Deraining: Optimization Model Driven Deep CNN. Proceedings of the 29th ACM International Conference on Multimedia (ACMM), Virtual.10.1145/3474085.3475441 ↗
18. Li, Y. (2016, January 27–30). Rain Streak Removal Using Layer Priors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.299 ↗
19. Zhu, L., Fu, C.W., Lischinski, D., and Heng, P.A. (2017, January 21–26). Joint Bi-layer Optimization for Single-Image Rain Streak Removal. Proceedings of the IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/ICCV.2017.276 ↗
20. Zhu, H., Wang, C., Zhang, Y., Su, Z., and Zhao, G. (2020, January 6–10). Physical Model Guided Deep Image Deraining. Proceedings of the 2020 IEEE International Conference on Multimedia and Expo (ICME), London, UK.10.1109/ICME46284.2020.9102878 ↗
21. Eigen, D., Krishnan, D., and Fergus, R. (2014, January 23–28). Restoring an Image Taken through a Window Covered with Dirt or Rain. Proceedings of the IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.10.1109/ICCV.2013.84 ↗
22. Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., and Yan, S. (2017, January 21–26). Deep Joint Rain Detection and Removal from a Single Image. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.183 ↗
23. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., and Paisley, J. (2017, January 21–26). Removing Rain from Single Images via a Deep Detail Network. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.186 ↗
24. Clearing the Skies: A Deep Network Architecture for SingleImage Rain Removal10.1109/TIP.2017.2691802 ↗
25. Li, G., He, X., Zhang, W., Chang, H., Dong, L., and Lin, L. (2018, January 22–26). Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining. Proceedings of the 26th ACM International Conference on Multimedia (ACMM), Seoul, Republic of Korea.10.1145/3240508.3240636 ↗
26. Fan, Z., Wu, H., Fu, X., Huang, Y., and Ding, X. (2018, January 22–26). Residual-Guide Network for Single Image Deraining. Proceedings of the 26th ACM International Conference on Multimedia (ACMM), Seoul, Republic of Korea.10.1145/3240508.3240694 ↗
27. Hu, X., Fu, C.W., Zhu, L., and Heng, P.A. (2019, January 15–20). Depth-Attentional Features for Single-Image Rain Removal. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00821 ↗
28. Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., and Lau, R.W. (2019, January 15–20). Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01255 ↗
29. Wang, C., Xing, X., Wu, Y., Su, Z., and Chen, J. (2020, January 12–16). DCSFN: Deep Cross-Scale Fusion Network for Single Image Rain Removal. Proceedings of the 28th ACM International Conference on Multimedia (ACMM), Seattle, WA, USA.10.1145/3394171.3413820 ↗
30. Jiang, K., Wang, Z., Yi, P., Chen, C., Huang, B., Luo, Y., Ma, J., and Jiang, J. (2020, January 13–19). Multi-Scale Progressive Fusion Network for Single Image Deraining. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00837 ↗
31. Deng, S., Wei, M., Wang, J., Feng, Y., Liang, L., Xie, H., Wang, F.L., and Wang, M. (2020, January 13–19). Detail-recovery Image Deraining via Context Aggregation Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01457 ↗
32. Wang, C., Wu, Y., Su, Z., and Chen, J. (2020, January 12–16). Joint Self-Attention and Scale-Aggregation for Self-Calibrated Deraining Network. Proceedings of the 28th ACM International Conference on Multimedia (ACMM), Seattle, WA, USA.10.1145/3394171.3413559 ↗
33. Multi-level Memory Compensation Network for Rain Removal via Divide-and-Conquer Strategy10.1109/JSTSP.2021.3052648 ↗
34. Deconvolution and Checkerboard Artifacts10.23915/distill.00003 ↗
35. A comparative analysis of early and late fusion for the multimodal two-class problem10.1109/ACCESS.2023.3296098 ↗
36. Graph Regularization Methods in Soft Detector Fusion10.1109/ACCESS.2023.3344776 ↗
37. Pereira, L.M., Salazar, A., and Vergara, L. (2023). A Comparative Study on Recent Automatic Data Fusion Methods. Computers, 13.10.3390/computers13010013 ↗
38. A proxy learning curve for the Bayes classifier10.1016/j.patcog.2022.109240 ↗
39. Image de-raining using a conditional generative adversarial network10.1109/TCSVT.2019.2920407 ↗
40. Zhang, H., and Patel, V.M. (2018, January 18–23). Density-aware single image de-raining using a multi-stream dense network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00079 ↗
41. Simoncelli. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
42. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
43. Loshchilov, I., and Hutter, F. (2017). SGDR: Stochastic gradient descent with warm restarts. arXiv.