Adaptive Prompting-Driven Degradation-Aware Fusion for Infrared and Visible Images
Source abstract
Abstract
Infrared and visible image fusion aims to combine the complementary advantages of thermal radiation information and rich texture details to generate more informative images for downstream perception tasks. However, existing deep learning-based methods usually assume ideal imaging conditions and often suffer from performance degradation in complex environments such as low illumination, rain interference, and strong lighting disturbances. To address this problem, this paper proposes an adaptive prompting-driven degradation-aware fusion framework. Specifically, a degradation-aware prompt generation module is introduced to automatically perceive degradation patterns from the input images and generate structured conditional prompts. These prompts guide the network to adaptively adjust feature representations through learnable affine modulation. Furthermore, a semantic-aligned feature learning strategy is designed to ensure consistent cross-modal representation in the latent space. Extensive experiments demonstrate that the proposed method achieves superior performance compared with several state-of-the-art fusion approaches under both normal and degraded conditions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Fusion Techniques · Image Enhancement Techniques · Infrared Target Detection Methodologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China
Funding
Natural Science Foundation of Shandong Province
References (68)
Show source-provided references
1. A survey on region based image fusion methods10.1016/j.inffus.2018.07.010 ↗
2. Deep learning-based multi-focus image fusion: A survey and a comparative study
3. Zhou, Y., Yu, L., Zhi, C., Huang, C., Wang, S., Zhu, M., and Fu, S. (2022). A survey of multi-focus image fusion methods. Appl. Sci., 12.10.3390/app12126281 ↗
4. Multi-modal 3D object detection in autonomous driving: A survey10.1007/s11263-023-01784-z ↗
5. Bao, C., Cao, J., Hao, Q., Cheng, Y., Ning, Y., and Zhao, T. (2023). Dual-YOLO architecture from infrared and visible images for object detection. Sensors, 23.10.3390/s23062934 ↗
6. Current advances and future perspectives of image fusion: A comprehensive review10.1016/j.inffus.2022.09.019 ↗
7. Ren, W., Luo, L., and Ren, J. (2025). A two-stage approach for infrared and visible image fusion and segmentation. Appl. Sci., 15.10.3390/app151910698 ↗
8. Minahil, S., Kim, J.H., and Hwang, Y. (2021). Patch-wise infrared and visible image fusion using spatial adaptive weights. Appl. Sci., 11.10.3390/app11199255 ↗
9. Hu, Y., Xu, S., Cheng, X., Zhou, C., and Xiong, M. (2023). AFSFusion: An adjacent feature shuffle combination network for infrared and visible image fusion. Appl. Sci., 13.10.3390/app13095640 ↗
10. PIAFusion: A progressive infrared and visible image fusion network based on illumination awareness10.1016/j.inffus.2022.03.007 ↗
11. Ma, J., Cheng, T., Wang, G., Zhang, Q., Wang, X., and Zhang, L. (2023). ProRes: Exploring degradation-aware visual prompt for universal image restoration. arXiv.
12. Textual prompt guided image restoration10.1016/j.engappai.2025.110981 ↗
13. Cui, Y., Zamir, S.W., Khan, S., Knoll, A., Shah, M., and Khan, F.S. (2025, January 24–28). Adair: Adaptive all-in-one image restoration via frequency mining and modulation. Proceedings of the International Conference on Learning Representations (ICLR), Singapore.
14. Yi, X., Xu, H., Zhang, H., Tang, L., and Ma, J. (2024, January 17–21). Text-IF: Leveraging semantic text guidance for degradation-aware and interactive image fusion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.02552 ↗
15. Wang, H., Zhang, H., Yi, X., Xiang, X., Fang, L., and Ma, J. (November, January 28). TERF: Text-driven and region-aware flexible visible and infrared image fusion. Proceedings of the 32nd ACM International Conference on Multimedia (ACM MM), Melbourne, VIC, Australia.
16. DenseFuse: A fusion approach to infrared and visible images10.1109/TIP.2018.2887342 ↗
17. RFN-Nest: An end-to-end residual fusion network for infrared and visible images10.1016/j.inffus.2021.02.023 ↗
18. LRRNet: A novel representation learning guided fusion network for infrared and visible images10.1109/TPAMI.2023.3268209 ↗
19. MUFusion: A general unsupervised image fusion network based on memory unit10.1016/j.inffus.2022.11.010 ↗
20. SEDRFuse: A symmetric encoder–decoder with residual block network for infrared and visible image fusion10.1109/TIM.2020.3022438 ↗
21. Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Xu, S., Lin, Z., and Van Gool, L. (2023, January 17–24). CDDFuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00572 ↗
22. FusionGAN: A generative adversarial network for infrared and visible image fusion10.1016/j.inffus.2018.09.004 ↗
23. DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion10.1109/TIP.2020.2977573 ↗
24. GANMcC: A generative adversarial network with multiclassification constraints for infrared and visible image fusion
25. Liu, J., Fan, X., Huang, Z., Wu, G., Liu, R., Zhong, W., and Luo, Z. (2022, January 18–24). Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00571 ↗
26. AttentionFGAN: Infrared and visible image fusion using attention-based generative adversarial networks10.1109/TMM.2020.2997127 ↗
27. Dual-attention-based feature aggregation network for infrared and visible image fusion
28. FusionGRAM: An infrared and visible image fusion framework based on gradient residual and attention mechanism
29. Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network10.1016/j.inffus.2021.12.004 ↗
30. CoCoNet: Coupled contrastive learning network with multi-level feature ensemble for multi-modality image fusion10.1007/s11263-023-01952-1 ↗
31. FAFusion: Learning for infrared and visible image fusion via frequency awareness10.1109/TIM.2024.3476526 ↗
32. Frequency integration and spatial compensation network for infrared and visible image fusion10.1016/j.inffus.2024.102359 ↗
33. Fu, Y., Xu, T., Wu, X., and Kittler, J. (2021). PPT fusion: Pyramid patch transformer for a case study in image fusion. arXiv.
34. YDTR: Infrared and visible image fusion via Y-shape dynamic transformer10.1109/TMM.2022.3192661 ↗
35. ITFuse: An interactive transformer for infrared and visible image fusion10.1016/j.patcog.2024.110822 ↗
36. DATFuse: Infrared and visible image fusion via dual attention transformer10.1109/TCSVT.2023.3234340 ↗
37. CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach10.1016/j.inffus.2023.102147 ↗
38. Cross-modal transformers for infrared and visible image fusion10.1109/TCSVT.2023.3289170 ↗
39. TCCFusion: An infrared and visible image fusion method based on transformer and cross correlation10.1016/j.patcog.2022.109295 ↗
40. AFT: Adaptive fusion transformer for visible and infrared images10.1109/TIP.2023.3263113 ↗
41. Infrared and low-light visible image fusion based on hybrid multiscale decomposition and adaptive light adjustment10.1016/j.optlaseng.2022.107268 ↗
42. He, C., Li, K., Xu, G., Zhang, Y., Hu, R., Guo, Z., and Li, X. (2023, January 1–6). Degradation-resistant unfolding network for heterogeneous image fusion. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.01159 ↗
43. A novel infrared and visible image fusion algorithm based on global information-enhanced attention network10.1016/j.imavis.2024.105161 ↗
44. A degradation-aware guided fusion network for infrared and visible image10.1016/j.inffus.2025.102931 ↗
45. Vaishnav, P., Syed Waqas, Z., Salman, K., and Fahad Shahbaz, K. (2023). PromptIR: Prompting for all-in-one blind image restoration. arXiv.
46. Li, Z., Lei, Y., Ma, C., Zhang, J., and Shan, H. (2023). Prompt-in-prompt learning for universal image restoration. SSRN, 6115113.
47. Qi, C., Tu, Z., Ye, K., Delbracio, M., Milanfar, P., Chen, Q., and Talebi, H. (October, January 29). SPIRE: Semantic prompt-driven image restoration. Proceedings of the European Conference on Computer Vision (ECCV), Milan, Italy.
48. Frequency domain nuances mining for visible-infrared person re-identification10.1109/TIFS.2025.3569176 ↗
49. Cong, X., Gui, J., Zhang, J., Hou, J., and Shen, H. (2024, January 16–22). A semi-supervised nighttime dehazing baseline with spatial-frequency aware and realistic brightness constraint. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00254 ↗
50. Loss functions for image restoration with neural networks10.1109/TCI.2016.2644865 ↗
51. Jia, X., Zhu, C., Li, M., Tang, W., and Zhou, W. (2021, January 11–17). LLVIP: A visible-infrared paired dataset for low-light vision. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCVW54120.2021.00389 ↗
52. U2Fusion: A unified unsupervised image fusion network10.1109/TPAMI.2020.3012548 ↗
53. The TNO multiband image data collection10.1016/j.dib.2017.09.038 ↗
54. Wang, Y., Miao, L., Zhou, Z., Zhang, L., and Qiao, Y. (2024). Infrared and visible image fusion with language-driven loss in CLIP embedding space. arXiv.10.1145/3746027.3755182 ↗
55. Liu, Z., Liu, J., Wu, G., Ma, L., Fan, X., and Liu, R. (2023). Bi-level dynamic learning for jointly multi-modality image fusion and beyond. arXiv.10.24963/ijcai.2023/138 ↗
56. Li, H., and Sun, Y. (2023, January 4–9). IGFNet: Illumination-guided fusion network for semantic scene understanding using RGB-thermal images. Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO), Koh Samui, Thailand.10.1109/ROBIO58561.2023.10354613 ↗
57. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
58. Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition10.1016/j.optcom.2014.12.032 ↗
59. Image quality measures and their performance10.1109/26.477498 ↗
60. In-fibre Bragg grating sensors10.1088/0957-0233/8/4/002 ↗
61. Information measure for performance of image fusion10.1049/el:20020212 ↗
62. Image information and visual quality10.1109/TIP.2005.859378 ↗
63. Making a “completely blind” image quality analyzer10.1109/LSP.2012.2227726 ↗
64. Venkatanath, N., Praneeth, D., Sumohana, S.C., and Swarup, S.M. (March, January 27). Blind image quality evaluation using perception based features. Proceedings of the National Conference on Communications (NCC), Mumbai, India.
65. A non-regularization self-supervised Retinex approach to low-light image enhancement with parameterized illumination estimation10.1016/j.patcog.2023.110025 ↗
66. Patil, P.W., Gupta, S., Rana, S., Venkatesh, S., and Murala, S. (2023, January 1–6). Multi-weather image restoration via domain translation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.01983 ↗
67. Li, B., Liu, X., Hu, P., Wu, Z., Lv, J., and Peng, X. (2022, January 18–24). All-in-one image restoration for unknown corruption. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01693 ↗
68. Cui, Z., Li, K., Gu, L., Su, S., Gao, P., Jiang, Z., and Harada, T. (2022). You only need 90k parameters to adapt light: A lightweight transformer for image enhancement and exposure correction. arXiv.