journal article·2026-05-10Open access

Deep Learning for Low-Light Vision: An Efficient Infrared–Visible Fusion Approach

Jiajie Lu, Viviana Desantis, Marco Brando Mario Paracchini, Marco Marcon

Applied Sciences· Volume 16· Issue 10

Abstract

Low-light enhancement technologies are of great significance for visual driver assistance applications and autonomous driving systems. Infrared vision can improve nighttime visibility but also faces challenges of low resolution and lack of color information. This paper presents a unified framework for RGB-guided infrared super-resolution and infrared-visible fusion that achieves high-resolution output under limited computational resources. Our approach employs a U-Net architecture with novel triple-grouped window attention (TGWA) encoding that captures global dependencies through grouped attention while reducing computational overhead, and adaptive multi-dilated convolutional (AMDC) decoding that adaptively selects optimal dilation rates using mixture-of-experts-inspired routing. Experiments on multiple datasets achieve competitive super-resolution and fusion results with minimal computational complexity, while real-world downstream object detection validation confirms robust performance in challenging nighttime scenarios. Quantitatively, the proposed method achieves 28.744 dB/0.872 SSIM on PBVS24 and 31.424 dB/0.882 SSIM on HDRT-Night for 8× infrared super-resolution, reaches competitive fusion quality on both MSRS and HDRT-Night, and attains 69.4% mAP@0.5 in downstream object detection on FLIR_aligned, while requiring only 1.12 M parameters and 85.44 G FLOPs. This work provides new possibilities for seeing clearly in the dark.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Image Enhancement Techniques · Advanced Neural Network Applications · Advanced Image Fusion Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, 20133 Milano, Italy

References (42)

Show source-provided references
1. Duda, R.O., Hart, P.E., and Stork, D.G. (2012). Pattern Classification, Wiley-Interscience.
2. Gonzalez, R.C., and Woods, R.E. (2018). Digital Image Processing, Pearson.
3. Method of interval fusion with preference aggregation in brightness thresholds selection for automatic weld surface defects recognition10.1016/j.measurement.2024.114969 ↗
4. Joint Bilateral Upsampling10.1145/1276377.1276497 ↗
5. Guided Image Filtering10.1109/TPAMI.2012.213 ↗
6. Deformable kernel networks for joint image filtering10.1007/s11263-020-01386-z ↗
7. Tang, J., Chen, X., and Zeng, G. (2021, January 20–24). Joint implicit image function for guided depth super-resolution. Proceedings of the ACM International Conference on Multimedia (ACM MM), Chengdu, China.10.1145/3474085.3475584 ↗
8. Song, X., Dai, Y., Zhou, D., Liu, L., Li, W., Li, H., and Yang, R. (2020, January 13–19). Channel attention based iterative residual learning for depth map super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00567 ↗
9. CODON: On orchestrating cross-domain attentions for depth super-resolution10.1007/s11263-021-01545-w ↗
10. Zhao, Z., Zhang, J., Xu, S., Lin, Z., and Pfister, H. (2022, January 18–24). Discrete cosine transform network for guided depth map super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00561 ↗
11. Yuan, J., Jiang, H., Li, X., Qian, J., Li, J., and Yang, J. (2023). Recurrent Structure Attention Guidance for Depth Super-Resolution. arXiv.10.1609/aaai.v37i3.25440 ↗
12. Wang, Z., Yan, Z., and Yang, J. (2024, January 20–27). Sgnet: Structure guided network via gradient-frequency awareness for depth map super-resolution. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.10.1609/aaai.v38i6.28395 ↗
13. Why thermal images are blurry10.1364/OE.506634 ↗
14. An Infrared Image Super-Resolution Imaging Algorithm Based on Auxiliary Convolution Neural Network
15. Kasliwal, A., Seth, P., Rallabandi, S., and Singhal, S. (2023, January 17–24). CoReFusion: Contrastive Regularized Fusion for Guided Thermal Super-Resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00057 ↗
16. Arnold, C., Jouvet, P., and Seoud, L. (2024, January 16–22). SwinFuSR: An image fusion-inspired model for RGB-guided thermal image super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.10.1109/CVPRW63382.2024.00308 ↗
17. Enhancement of guided thermal image super-resolution approaches10.1016/j.neucom.2023.127197 ↗
18. Infrared and Visible Image Fusion via Gradient Transfer and Total Variation Minimization10.1016/j.inffus.2016.02.001 ↗
19. FusionGAN: A Generative Adversarial Network for Infrared and Visible Image Fusion10.1016/j.inffus.2018.09.004 ↗
20. U2Fusion: A Unified Unsupervised Image Fusion Network10.1109/TPAMI.2020.3012548 ↗
21. SDNet: A Versatile Squeeze-and-Decomposition Network for Real-Time Image Fusion10.1007/s11263-021-01501-8 ↗
22. PIAFusion: A Progressive Infrared and Visible Image Fusion Network Based on Illumination Aware10.1016/j.inffus.2022.03.007 ↗
23. Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network10.1016/j.inffus.2021.12.004 ↗
24. Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Xu, S., Lin, Z., Timofte, R., and Van Gool, L. (2023, January 17–24). CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00572 ↗
25. ITFuse: An Interactive Transformer for Infrared and Visible Image Fusion10.1016/j.patcog.2024.110822 ↗
26. Li, A., Yin, G., Wang, Z., Liang, J., Wang, F., Bai, X., and Liu, Z. (2024, January 2–5). RCAFusion: Cross Rubik Cube Attention Network for Multi-modal Image Fusion of Intelligent Vehicles. Proceedings of the 2024 IEEE Intelligent Vehicles Symposium (IV), Jeju Island, Republic of Korea.10.1109/IV55156.2024.10588756 ↗
27. Liu, J., Fan, X., Huang, Z., Wu, G., Liu, R., Zhong, W., and Luo, Z. (2022, January 18–24). Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00571 ↗
28. A Deep Learning Framework for Infrared and Visible Image Fusion Without Strict Registration10.1007/s11263-023-01948-x ↗
29. Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Springer International Publishing.10.1007/978-3-319-24571-3 ↗
30. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems, Curran Associates, Inc.
31. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Virtual.10.1109/ICCV48922.2021.00986 ↗
32. Adaptive Mixtures of Local Experts10.1162/neco.1991.3.1.79 ↗
33. Lepikhin, D., Lee, H., Xu, Y., Chen, D., Firat, O., Huang, Y., Krikun, M., Shazeer, N., and Chen, Z. (2020). Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv.
34. Chen, L., Chu, X., Zhang, X., and Sun, J. (2022, January 23–27). Simple Baselines for Image Restoration. Proceedings of the European Conference on Computer Vision (ECCV), Tel Aviv, Israel.10.1007/978-3-031-20071-7_2 ↗
35. Rivadeneira, R.E., Sappa, A.D., Wang, C., Jiang, J., Zhong, Z., Chen, P., and Wang, S. (2024, January 16–22). Thermal Image Super-Resolution Challenge Results-PBVS 2024. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.10.1109/CVPRW63382.2024.00317 ↗
36. HDRT: A Large-Scale Dataset for Infrared-Guided HDR Imaging10.1016/j.inffus.2025.103109 ↗
37. Zhang, H., Fromont, E., Lefevre, S., and Avignon, B. (2020, January 25–28). Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.10.1109/ICIP40778.2020.9191080 ↗
38. Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., and Timofte, R. (2021, January 11–17). SwinIR: Image Restoration Using Swin Transformer. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada.10.1109/ICCVW54120.2021.00210 ↗
39. Deep Attentional Guided Image Filtering10.1109/TNNLS.2023.3253472 ↗
40. Shi, W., Ye, M., and Du, B. (2022, January 10–14). Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution. Proceedings of the 30th ACM International Conference on Multimedia, Lisboa, Portugal.10.1145/3503161.3547873 ↗
41. SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer10.1109/JAS.2022.105686 ↗
42. Jocher, G., Qiu, J., and Chaurasia, A. (2026, April 17). Ultralytics YOLO. GitHub Repository. 2024. Version 11.0.0. Available online: https://github.com/ultralytics/ultralytics.