journal article·2024Open access

Joint Rain Streaks & Haze Removal Network for Object Detection

Ragini Thatikonda, Prakash Kodali, Ramalingaswamy Cheruku, Eswaramoorthy K.V

Computers, Materials & Continua· Volume 79· Issue 3

Abstract

In the realm of low-level vision tasks, such as image deraining and dehazing, restoring images distorted by adverse weather conditions remains a significant challenge. The emergence of abundant computational resources has driven the dominance of deep Convolutional Neural Networks (CNNs), supplanting traditional methods reliant on prior knowledge. However, the evolution of CNN architectures has tended towards increasing complexity, utilizing intricate structures to enhance performance, often at the expense of computational efficiency. In response, we propose the Selective Kernel Dense Residual M-shaped Network (SKDRMNet), a flexible solution adept at balancing computational efficiency with network accuracy. A key innovation is the incorporation of an M-shaped hierarchical structure, derived from the U-Net framework as M-Network (M-Net), within which the Selective Kernel Dense Residual Module (SDRM) is introduced to reinforce multi-scale semantic feature maps. Our methodology employs two sampling techniques-bilinear and pixel unshuffled and utilizes a multi-scale feature fusion approach to distil more robust spatial feature map information. During the reconstruction phase, feature maps of varying resolutions are seamlessly integrated, and the extracted features are effectively merged using the Selective Kernel Fusion Module (SKFM). Empirical results demonstrate the comprehensive superiority of SKDRMNet across both synthetic and real rain and haze datasets.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Image Enhancement Techniques · Fire Detection and Safety Systems · Video Surveillance and Tracking Methods

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (56)

Show source-provided references
1. SSAD: Single-shot multi-scale attentive detector for autonomous driving10.1080/02564602.2023.2176932 ↗
2. G. Jocher, A. Chaurasia, and J. Qiu, “Yolo by ultralytics (version 8.0.190),” GitHub, 2023. Accessed: Apr. 1, 2024. [Online]. Available: https://github.com/ultralytics/ultralytics.
3. YOLOv2PD: An efficient pedestrian detection algorithm using improved YOLOv2 model
4. Single image rain and snow removal via guided L0 smoothing filter10.1007/s11042-015-2657-7 ↗
5. J. Xu, W. Zhao, P. Liu, and X. Tang, “Removing rain and snow in a single image using guided filter,” presented at the 2012 IEEE Int. Conf. Comp. Sci. Autom. Eng. (CSAE), Zhangjiajie, China, May 25–27, 2012.10.1109/CSAE.2012.6272780 ↗
6. X. Zheng, Y. Liao, W. Guo, X. Fu, and X. Ding, “Single-image-based rain and snow removal using multi-guided filter,” presented at the 2013 Neural Inform. Process. 20th Int. Conf. (ICONIP), Daegu, Korea, Nov. 3–7, 2013.
7. Y. Li, R. T. Tan, X. Guo, J. Lu, and M. S. Brown, “Rain streak removal using layer priors,” presented at the 2016 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Las Vegas, NV, USA, Jun. 27–30, 2016.10.1109/CVPR.2016.299 ↗
8. Y. L. Chen and C. T. Hsu, “A generalized low-rank appearance model for spatio-temporally correlated rain streaks,” presented at the 2013 Proc. IEEE Int. Conf. Comp. Vis. (ICCV), Washington DC, USA, Dec. 1–8, 2013.10.1109/ICCV.2013.247 ↗
9. R. Qian, R. T. Tan, W. Yang, J. Su, and J. Liu, “Attentive generative adversarial network for raindrop removal from a single image,” presented at the 2018 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Salt Lake City, UT, USA, Jun. 18–23, 2018.10.1109/CVPR.2018.00263 ↗
10. Single image haze removal using dark channel prior
11. X. Li, J. Wu, Z. Lin, H. Liu, and H. Zha, “Recurrent squeeze-and-excitation context aggregation net for single image deraining,” presented at the 2018 Proc. Euro. Conf. Comp. Vis. (ECCV), Munich, Germany, Sep. 8–14, 2018.10.1007/978-3-030-01234-2_16 ↗
12. S. W. Zamir et al., “Multi-stage progressive image restoration,” presented at the 2021 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Nashville, TN, USA, Jun. 20–25, 2021.10.1109/CVPR46437.2021.01458 ↗
13. S. Zheng, C. Lu, Y. Wu, and G. Gupta, “SAPNet: Segmentation-aware progressive network for perceptual contrastive deraining,” presented at the 2022 Proc. IEEE Win. Conf. App’s. Comp. Vis., Waikoloa, HI, USA, Jan. 4–8, 2022.10.1109/WACVW54805.2022.00011 ↗
14. C. Mou, Q. Wang, and J. Zhang, “Deep generalized unfolding networks for image restoration,” presented at the 2022 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), New Orleans, LA, USA, Jun. 18–24, 2022.10.1109/CVPR52688.2022.01688 ↗
15. Deep feature interactive aggregation network for single image deraining10.1109/ACCESS.2022.3210190 ↗
16. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” presented at the 2016 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Las Vegas, NV, USA, Jun. 27–30, 2016.10.1109/CVPR.2016.90 ↗
17. X. Fu, J. Huang, D. Zeng, Y. Huang, X. Ding, and J. Paisley, “Removing rain from single images via a deep detail network,” presented at the 2017 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Honolulu, Hawaii, USA, Jul. 21–26, 2017.10.1109/CVPR.2017.186 ↗
18. Single image rain removal via multi-module deep grid network10.1016/j.cviu.2020.103106 ↗
19. Exploring overcomplete representations for single image deraining using cnns10.1109/JSTSP.2020.3039393 ↗
20. W. Yang, R. T. Tan, J. Feng, J. Liu, Z. Guo and S. Yan, “Deep joint rain detection and removal from a single image,” presented at the 2017 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Honolulu, Hawaii, USA, Jul. 21–26, 2017.10.1109/CVPR.2017.183 ↗
21. H. Zhang and V. M. Patel, “Density-aware single image de-raining using a multi-stream dense network,” presented at the 2018 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Salt Lake City, UT, USA, Jun. 18–23, 2018.10.1109/CVPR.2018.00079 ↗
22. X. Qin, Z. Wang, Y. Bai, X. Xie, and H. Jia, “FFA-Net: Feature fusion attention network for single image dehazing,” presented at the 2020 Proc. AAAI Conf. Artif. Intel., New York, USA, Apr. 3–5, 2020.
23. Lightweight semi-supervised network for single image rain removal10.1016/j.patcog.2022.109277 ↗
24. W. Wei, D. Meng, Q. Zhao, Z. Xu, and Y. Wu, “Semi-supervised transfer learning for image rain removal,” presented at the 2019 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Long Beach, CA, USA, Jun. 15–20, 2019.10.1109/CVPR.2019.00400 ↗
25. FPD-M-net: Fingerprint image denoising and in painting using M-net based convolutional neural networks10.1007/978-3-030-25614-2_4 ↗
26. C. M. Fan, T. J. Liu, K. H. Liu, and C. H. Chiu, “Selective residual M-net for real image denoising,” presented at the 2022 30th Euro. Signal Process. Conf. (EUSIPCO), Belgrade, Serbia, Aug. 29–Sep. 2, 2022, pp. 469–473.10.23919/EUSIPCO55093.2022.9909521 ↗
27. S. W. Zamir et al., “Learning enriched features for real image restoration and enhancement,” presented at the 2020 16th Euro. Conf. Comp. Vis. (ECCV), Glasgow, UK, Aug. 23–28, 2020.
28. Video deraining and desnowing using temporal correlation and low-rank matrix completion10.1109/TIP.2015.2428933 ↗
29. K. Garg and S. K. Nayar, “Detection and removal of rain from videos,” presented at the 2004 Proc. IEEE Comp. Society Conf. Comp. Vis. Pattern Rec. (CVPR), Washington DC, USA, Jun. 27– Jul. 02, 2004.
30. D3R-Net: Dynamic routing residue recurrent network for video rain removal10.1109/TIP.2018.2869722 ↗
31. Image de-raining using a conditional generative adversarial network10.1109/TCSVT.2019.2920407 ↗
32. Y. Wang, Y. Song, C. Ma, and B. Zeng, “Rethinking image deraining via rain streaks and vapors,” presented at the 2020 Proc. 16th Euro. Conf. Comp. Vis. (ECCV), Glasgow, UK, Aug. 23–28, 2020.10.1007/978-3-030-58520-4_22 ↗
33. R. Li, L. F. Cheong, and R. T. Tan, “Heavy rain image restoration: Integrating physics model and conditional adversarial learning,” presented at the 2019 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Long Beach, CA, USA, Jun. 15–20, 2019.10.1109/CVPR.2019.00173 ↗
34. Y. Li, Y. Monno, and M. Okutomi, “Single image deraining network with rain embedding consistency and layered lstm,” presented at the 2022 Proc. IEEE Winter Conf. Apps Comp. Vis. (CACV), Waikoloa, HI, USA, Jan. 3–8, 2022.10.1109/WACV51458.2022.00401 ↗
35. Multi-scale progressive fusion network for single image deraining
36. Clearing the skies: A deep network architecture for single-image rain removal10.1109/TIP.2017.2691802 ↗
37. D. Ren, W. Zuo, Q. Hu, P. Zhu, and D. Meng, “Progressive image deraining networks: A better and simpler baseline,” presented at the 2019 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Long Beach, CA, USA, Jun. 15–20, 2019.10.1109/CVPR.2019.00406 ↗
38. C. Chen and H. Li, “Robust representation learning with feedback for single image deraining,” presented at the 2021 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Nashville, TN, USA, Jun. 20–25, 2021.10.1109/CVPR46437.2021.00765 ↗
39. H. Lee, H. Choi, K. Sohn, and D. Min, “KNN local attention for image restoration,” presented at the 2022 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), New Orleans, LA, USA, Jun. 18–24, 2022.10.1109/CVPR52688.2022.00218 ↗
40. Image deraining transformer10.1109/TPAMI.2022.3183612 ↗
41. PFDN: Pyramid feature decoupling network for single image deraining10.1109/TIP.2022.3219227 ↗
42. Recurrent context-aware multi-stage network for single image deraining10.1016/j.cviu.2022.103612 ↗
43. Single image deraining: From model-based to data-driven and beyond10.1109/TPAMI.2020.2995190 ↗
44. H. Wu et al., “Contrastive learning for compact single image dehazing,” presented at the 2021 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Nashville, TN, USA, Jun. 20–25, 2021.10.1109/CVPR46437.2021.01041 ↗
45. C. L. Guo, Q. Yan, S. Anwar, R. Cong, W. Ren, and C. Li, “Image dehazing transformer with transmission-aware 3d position embedding,” presented at the 2022 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), New Orleans, LA, USA, Jun. 18–24, 2022.10.1109/CVPR52688.2022.00572 ↗
46. R. Q. Wu, Z. P. Duan, C. L. Guo, Z. Chai, and C. Li, “RIDCP: Revitalizing real image dehazing via high-quality codebook priors,” presented at the 2023 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Vancouver, BC, Canada, Jun. 17–24, 2023.
47. H. Yu, N. Zheng, M. Zhou, J. Huang, Z. Xiao and F. Zhao, “Frequency and spatial dual guidance for image dehazing,” presented at the 2022 Euro. Conf. Comp. Vis. (ECCV), Tel Aviv, Israel, Oct. 23–27, 2022.10.1007/978-3-031-19800-7_11 ↗
48. Frequency and content dual stream network for image dehazing10.1016/j.imavis.2023.104820 ↗
49. FSAD-Net: Feedback spatial attention dehazing network10.1109/TNNLS.2022.3146004 ↗
50. TUSR-Net: Triple unfolding single image dehazing with self-regularization and dual feature to pixel attention10.1109/TIP.2023.3234701 ↗
51. R. Mehta and J. Sivaswamy, “M-net: A convolutional neural network for deep brain structure segmentation,” presented at the 2017 IEEE 14th Int. Sym. Biomed. Imag. (ISBI), Melbourne, Australia, Apr. 18–21, 2017.10.1109/ISBI.2017.7950555 ↗
52. Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, “Residual dense network for image super-resolution,” presented at the 2018 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Salt Lake City, UT, USA, Jun. 18–23, 2018.10.1109/CVPR.2018.00262 ↗
53. S. Li et al., “Single image deraining: A comprehensive benchmark analysis,” presented at the 2019 Proc. IEEE Conf. Comp. Vis. Pattern Rec. (CVPR), Long Beach, CA, USA, Jun. 15–20, 2019.10.1109/CVPR.2019.00396 ↗
54. Benchmarking single image dehazing and beyond10.1109/TIP.2018.2867951 ↗
55. C. O. Ancuti, C. Ancuti, M. Sbert, and R. Timofte, “Dense-Haze: A benchmark for image dehazing with Dense-Haze and haze-free images,” presented at the 2019 IEEE Int. Conf. Image Pro. (ICIP), Taipei, Taiwan, Sep. 22–25, 2019.10.1109/ICIP.2019.8803046 ↗
56. C. O. Ancuti, C. Ancuti, and R. Timofte, “NH-HAZE: An image dehazing benchmark with non-homogeneous hazy and haze-free images,” presented at the 2020 Proc. EEE/CVF Conf. Com. Vis. Pattern Rec. Workshops (CVPRW), Seattle, WA, USA, Jun. 14–19, 2020.10.1109/CVPRW50498.2020.00230 ↗