SnowMamba: Achieving More Precise Snow Removal with Mamba
Source abstract
Abstract
Due to the diversity and semi-transparency of snowflakes, accurately locating and reconstructing background information during image restoration poses a significant challenge. Snowflakes obscure image details, thereby affecting downstream tasks such as object recognition and image segmentation. Although Convolutional Neural Networks (CNNs) and Transformers have achieved promising results in snow removal through local or global feature processing, residual snowflakes or shadows persist in restored images. Inspired by the recent popularity of State Space Models (SSMs), this paper proposes a Mamba-based multi-scale desnowing network (SnowMamba), which effectively models the long-range dependencies of snowflakes. This enables the precise localization and removal of snow particles, addressing the issue of residual snowflakes and shadows in images. Specifically, we design a four-stage encoder–decoder network that incorporates Snow Caption Mamba (SCM) and SE modules to extract comprehensive snowflake and background information. The extracted multi-scale snow and background features are then fed into the proposed Multi-Scale Residual Interaction Network (MRNet) to learn and reconstruct clear, snow-free background images. Extensive experiments demonstrate that the proposed method outperforms other mainstream desnowing approaches in both qualitative and quantitative evaluations on three standard image desnowing datasets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Winter Sports Injuries and Performance · Cryospheric studies and observations · Computer Graphics and Visualization Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Technology, Xinjiang University, Urumqi 830046, China
Key Laboratory of Signal Detection and Processing, Xinjiang University, Urumqi 830046, China
Funding
Natural Science Foundation of XinJiang province · National Natural Science Foundation of China · Tianshan Talent Training Project—Xinjiang Science and Technology Innovation Team Program · Tianshan Innovation Team Program of Xinjiang Uygur Autonomous Region of China · Scientific Research Plan of Universities of Xinjiang
References (31)
Show source-provided references
1. Zhao, Y., Lv, W., Xu, S., Wei, J., Wang, G., Dang, Q., Liu, Y., and Chen, J. (2024, January 16–22). Detrs beat yolos on real-time object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.01605 ↗
2. Liu, Q., Xu, Z., Bertasius, G., and Niethammer, M. (2023, January 1–6). Simpleclick: Interactive image segmentation with simple vision transformers. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.02037 ↗
3. A survey on deep learning-based real-time crowd anomaly detection for secure distributed video surveillance10.1007/s00779-021-01586-5 ↗
4. Zheng, X., Liao, Y., Guo, W., Fu, X., and Ding, X. (2013, January 3–7). Single-image-based rain and snow removal using multi-guided filter. Proceedings of the Neural Information Processing: 20th International Conference, ICONIP 2013, Daegu, Republic of Korea. Proceedings, Part III 20.
5. Single image rain and snow removal via guided L0 smoothing filter10.1007/s11042-015-2657-7 ↗
6. A hierarchical approach for rain or snow removing in a single color image10.1109/TIP.2017.2708502 ↗
7. DesnowNet: Context-aware deep network for snow removal10.1109/TIP.2018.2806202 ↗
8. Stacked dense networks for single-image snow removal10.1016/j.neucom.2019.07.023 ↗
9. Chen, W.T., Fang, H.Y., Ding, J.J., Tsai, C.C., and Kuo, S.Y. (2020, January 23–28). JSTASR: Joint size and transparency-aware snow removal algorithm based on modified partial convolution and veiling effect removal. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part XXI 16.10.1007/978-3-030-58589-1_45 ↗
10. Chen, W.T., Fang, H.Y., Hsieh, C.L., Tsai, C.C., Chen, I., Ding, J.J., and Kuo, S.Y. (2021, January 10–17). All snow removed: Single image desnowing algorithm using hierarchical dual-tree complex wavelet representation and contradict channel loss. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00416 ↗
11. Lightweight image de-snowing: A better trade-off between network capacity and performance10.1016/j.neunet.2023.06.029 ↗
12. Snow mask guided adaptive residual network for image snow removal10.1016/j.cviu.2023.103819 ↗
13. Image desnowing via deep invertible separation10.1109/TCSVT.2022.3233655 ↗
14. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
15. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., and Yang, M.H. (2022, January 18–24). Restormer: Efficient transformer for high-resolution image restoration. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00564 ↗
16. Image de-raining transformer10.1109/TPAMI.2022.3183612 ↗
17. Chen, W.T., Huang, Z.K., Tsai, C.C., Yang, H.H., Ding, J.J., and Kuo, S.Y. (2022, January 18–24). Learning multiple adverse weather removal via two-stage knowledge learning and multi-contrastive regularization: Toward a unified model. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01713 ↗
18. LMQFormer: A laplace-prior-guided mask query transformer for lightweight snow removal10.1109/TCSVT.2023.3264824 ↗
19. Zhang, T., Jiang, N., Wu, H., Zhang, K., Niu, Y., and Zhao, T. (November, January 29). HCSD-Net: Single Image Desnowing with Color Space Transformation. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada.
20. Chen, S., Ye, T., Liu, Y., Liao, T., Jiang, J., Chen, E., and Chen, P. (2023, January 4–10). Msp-former: Multi-scale projection transformer for single image desnowing. Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece.10.1109/ICASSP49357.2023.10095605 ↗
21. Gu, A., and Dao, T. (2023). Mamba: Linear-time sequence modeling with selective state spaces. arXiv.
22. Ma, J., Li, F., and Wang, B. (2024). U-mamba: Enhancing long-range dependency for biomedical image segmentation. arXiv.
23. Wang, Z., Zheng, J.Q., Zhang, Y., Cui, G., and Li, L. (2024). Mamba-unet: Unet-like pure visual mamba for medical image segmentation. arXiv.10.2139/ssrn.5097998 ↗
24. Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W., and Wang, X. (2024). Vision mamba: Efficient visual representation learning with bidirectional state space model. arXiv.
25. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
26. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
27. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–23). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗
28. Image quality assessment: Unifying structure and texture similarity
29. Li, R., Tan, R.T., and Cheong, L.F. (2020, January 13–19). All in one bad weather removal using architectural search. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00324 ↗
30. Valanarasu, J.M.J., Yasarla, R., and Patel, V.M. (2022, January 18–24). Transweather: Transformer-based restoration of images degraded by adverse weather conditions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.
31. Chen, E., Chen, S., Ye, T., and Liu, Y. (2024). Degradation-adaptive neural network for jointly single image dehazing and desnowing. Front. Comput. Sci., 18.10.1007/s11704-023-2764-y ↗