Multi-Scale Aggregation Residual Channel Attention Fusion Network for Single Image Deraining
Source abstract
Abstract
Images captured on rainy days are prone to rain streaking on various scales. These images taken on a rainy day will be disturbed by rain streaks of varying degrees, resulting in degradation of image quality. This study sought to eliminate rain streaks from images using a two-stage network architecture involving progressive multi-scale recovery and aggregation. The proposed multi-scale aggregation residual channel attention fusion network (MARCAFNet) uses kernels of various scales to recover details at various levels of granularity to enhance the robustness of the model to streaks of various sizes, densities, and shapes. When applied to benchmark datasets, the proposed method outperformed other state-of-the-art schemes in the restoration of image details without distorting the image structure.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Image Enhancement Techniques · Advanced Image Fusion Techniques · Advanced Image Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
The Department of Medical Informatics, Tzu Chi University, Hualien County 97004, Taiwan
701 Zhongyang Rd., Sec. 3, Hualien County 97004, Taiwan
References (43)
Show source-provided references
1. Sun, Z., Bebis, G., and Miller, R. (2002, January 1–3). On-road vehicle detection using Gabor filters and support vector machines. Proceedings of the International Conference on Digital Signal Processing, Santorini, Greece.
2. Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art10.1561/0600000079 ↗
3. Automatic single-image-based rain streaks removal via image decomposition10.1109/TIP.2011.2179057 ↗
4. A hierarchical approach for rain or rain or snow removing in a single color image10.1109/TIP.2017.2708502 ↗
5. Li, Y., Tan, R.T., Guo, X., Lu, J., and Brown, M.S. (2016, January 27–30). Rain streak removal using layer priors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.299 ↗
6. Yu, L., Yong, X., and Hui, J. (2015, January 7–13). Removing rain from a single image via discriminative sparse coding. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.
7. Li, X., Wu, J., Lin, Z., Liu, H., and Zha, H. (2018, January 8–14). Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01234-2_16 ↗
8. Ren, D., Zuo, W., Hu, Q., Zhu, P., and Meng, D. (2019, January 20–25). Progressive image deraining networks: A better and simpler baseline. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00406 ↗
9. Lightweight pyramid networks for image deraining10.1109/TNNLS.2019.2926481 ↗
10. Jiang, K., Wang, Z., Yi, P., Chen, C., Huang, B., Luo, Y., Ma, J., and Jiang, J. (2020, January 14–19). Multi-Scale Progressive Fusion Network for Single Image Deraining. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00837 ↗
11. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H., and Shao, L. (2021, January 19–25). Multi-Stage Progressive Image Restoration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01458 ↗
12. Wang, H., Yue, Z., Xie, Q., Zhao, Q., Zheng, Y., and Meng, D. (2021, January 19–25). From rain generation to rain. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01455 ↗
13. Rain-Free and Residue Hand-in-Hand: A Progressive Coupled Network for Real-Time Image Deraining10.1109/TIP.2021.3102504 ↗
14. RAiA-Net: A Multi-Stage Network with Refined Attention in Attention Module for Single Image Deraining10.1109/LSP.2022.3154981 ↗
15. Multi-scale analysis strategies in PRNU-based tampering localization10.1109/TIFS.2016.2636089 ↗
16. Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., and Cottrell, G. (2018, January 12–15). Understanding Convolution for Semantic Segmentation. Proceedings of the IEEE Winter Conference, Lake Tahoe, NV, USA.10.1109/WACV.2018.00163 ↗
17. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
18. Huang, G., Liu, Z., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
19. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
20. Garg, K., and Nayar, S.K. (July, January 27). Detection and removal of rain from videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA.
21. Zhang, X., Li, H., Qi, Y., Leow, W.K., and Ng, T.K. (2006, January 9–12). Rain removal in video by combining temporal and chromatic properties. Proceedings of the IEEE International Conference on Multimedia and Expo, Hilton Toronto, ON, Canada.10.1109/ICME.2006.262572 ↗
22. Chen, Y.L., and Hsu, C.T. (2013, January 1–8). A generalized low-rank appearance model for spatio-temporally correlated rain streaks. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.247 ↗
23. Video deraining and desnowing using temporal correlation and low-rank matrix completion10.1109/TIP.2015.2428933 ↗
24. Li, M., Xie, Q., Zhao, Q., Wei, W., Gu, S., Tao, J., and Meng, D. (2018, January 18–22). Video rain streak removal by Multiscale Convolutional sparse coding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00695 ↗
25. Liu, J., Yang, W., Yang, S., and Guo, Z. (2018, January 18–22). Erase of fill? Deep joint recurrent rain removal and reconstruction in videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00341 ↗
26. Yue, Z., Xie, J., Zhao, Q., and Meng, D. (2021, January 19–25). Semi-Supervised Video Deraining with Dynamical Rain Generator. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00070 ↗
27. Self-learning based image decomposition with applications to single image denoising10.1109/TMM.2013.2284759 ↗
28. Zhu, L., Fu, C.W., Lischinski, D., and Heng, P.A. (2017, January 22–29). Joint bi-layer optimization for single-image rain streak removal Deep-learning. Proceedings of the International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.276 ↗
29. Eigen, D., Krishnan, D., and Fergus, R. (2013, January 1–8). Restoring an image taken through a window covered with dirt or rain. Proceedings of the International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.84 ↗
30. Clearing the skies: A deep network architecture for single-image rain removal10.1109/TIP.2017.2691802 ↗
31. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., and Paisley, J. (2017, January 21–26). Removing rain from single images via a deep detail network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.186 ↗
32. Hu, X., Fu, C.W., Zhu, L., and Heng, P.A. (2019, January 20–25). Depth-attentional features for single-image rain removal. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00821 ↗
33. Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., and Lau, R.W. (2019, January 20–25). Spatial attentive single-image deraining with a high quality real rain dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.01255 ↗
34. Image de-raining using a conditional generative adversarial network10.1109/TCSVT.2019.2920407 ↗
35. Chen, L., Lu, X., Zhang, J., Chu, X., and Chen, C. (2021, January 19–25). HINet: Half Instance Normalization Network for Image Restoration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Nashville, TN, USA.10.1109/CVPRW53098.2021.00027 ↗
36. PFDN: Pyramid Feature Decoupling Network for Single Image Deraining10.1109/TIP.2022.3219227 ↗
37. Yu, F., and Koltun, V. (2016, January 2–4). Multi-Scale Context Aggregation by Dilated Convolutions. Proceedings of the International Conference on Learning Representations, San Juan, Puerto Rico.
38. Maas, A.L., Hannum, A.Y., and Ng, A.Y. (2013, January 16–21). Rectifier nonlinearities improve neural network acoustic models. Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA.
39. Hu, J.L., Albanie, S., Sun, G., and Vedaldi, A. (2018, January 18–22). Squeeze-and-Excitation Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
40. He, K., Sun, J., and Tang, X. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
41. Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., and Yan, S. (2017, January 21–26). Deep joint rain detection and removal from a single image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.183 ↗
42. Kingma, D., and Ba, J. (2015, January 7–9). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA.
43. Scope of validity of psnr in image/video quality assessment10.1049/el:20080522 ↗