D3Fusion: Decomposition–Disentanglement–Dynamic Compensation Framework for Infrared-Visible Image Fusion in Extreme Low-Light
Source abstract
Abstract
Infrared-visible image fusion quality is critical for nighttime perception in autonomous driving and surveillance but suffers severe degradation under extreme low-light conditions, including irreversible texture loss in visible images, thermal boundary diffusion artifacts, and overexposure under dynamic non-uniform illumination. To address these challenges, a Decomposition–Disentanglement–Dynamic Compensation framework, D3Fusion, is proposed. Firstly, a Retinex-inspired Decomposition Illumination Net (DIN) decomposes inputs into enhanced images and degradative illumination maps for joint low-light recovery. Secondly, an illumination-guided encoder and a multi-scale differential compensation decoder dynamically balance cross-modal features. Finally, a progressive three-stage training paradigm from illumination correction through feature disentanglement to adaptive fusion resolves optimization conflicts. Compared to State-of-the-Art methods, on the LLVIP, TNO, MSRS, and RoadScene datasets, D3Fusion achieves an average improvement of 1.59% in standard deviation (SD), 6.9% in spatial frequency (SF), 2.59% in edge intensity (EI), and 1.99% in visual information fidelity (VIF), demonstrating superior performance in extreme low-light scenarios. The framework effectively suppresses thermal diffusion artifacts while mitigating exposure imbalance, adaptively brightening scenes while preserving texture details in shadowed regions. This significantly improves fusion quality for nighttime images by enhancing salient information, establishing a robust solution for multimodal perception under illumination-critical conditions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Fusion Techniques · Image Enhancement Techniques · Visual Attention and Saliency Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechanical Engineering, Sichuan University, Chengdu 610065, China
School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
Funding
National Natural Science Foundation of China · Natural Science Foundation of Sichuan Province of China
References (31)
Show source-provided references
1. DIVFusion: Darkness-free infrared and visible image fusion10.1016/j.inffus.2022.10.034 ↗
2. Visible and Infrared Image Fusion Using Deep Learning10.1109/TPAMI.2023.3261282 ↗
3. PIAFusion: A progressive infrared and visible image fusion network based on illumination aware10.1016/j.inffus.2022.03.007 ↗
4. Jin, Y., Yang, W., and Tan, R.T. (2022, January 23–27). Unsupervised Night Image Enhancement: When Layer Decomposition Meets Light-Effects Suppression. Proceedings of the 17th European Conference on Computer Vision (ECCV), Tel Aviv, Israel.10.1007/978-3-031-19836-6_23 ↗
5. WaveletFormerNet: A Transformer-based wavelet network for real-world non-homogeneous and dense fog removal10.1016/j.imavis.2024.105014 ↗
6. CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach10.1016/j.inffus.2023.102147 ↗
7. Lightness and Retinex Theory10.1364/JOSA.61.000001 ↗
8. Naturalness Preserved Enhancement Algorithm for Non-Uniform Illumination Images10.1109/TIP.2013.2261309 ↗
9. Fu, X.Y., Zeng, D.L., Huang, Y., Zhang, X.P., and Ding, X.H. (2016, January 27–30). A weighted variational model for simultaneous reflectance and illumination estimation. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR.2016.304 ↗
10. Properties and performance of a center/surround retinex10.1109/83.557356 ↗
11. A multiscale retinex for bridging the gap between color images and the human observation of scenes10.1109/83.597272 ↗
12. Zhang, Y.H., Zhang, J.W., and Guo, X.J. (2019, January 21–25). Kindling the Darkness: A Practical Low-light Image Enhancer. Proceedings of the 27th ACM International Conference on Multimedia (MM), Nice, France.10.1145/3343031.3350926 ↗
13. Cai, Y., Bian, H., Lin, J., Wang, H., Timofte, R., and Zhang, Y. (2023, January 2–6). Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancement. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.01149 ↗
14. Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model10.1109/TIP.2018.2810539 ↗
15. Beyond Brightening Low-light Images10.1007/s11263-020-01407-x ↗
16. FCLFusion: A frequency-aware and collaborative learning for infrared and visible image fusion10.1016/j.engappai.2024.109192 ↗
17. U2Fusion: A Unified Unsupervised Image Fusion Network10.1109/TPAMI.2020.3012548 ↗
18. SwinFusion: Cross-domain Long-range Learning for General Image Fusion via Swin Transformer10.1109/JAS.2022.105686 ↗
19. Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Xu, S., Lin, Z., Timofte, R., and Van Gool, L. (2023, January 17–24). CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00572 ↗
20. FusionGAN: A generative adversarial network for infrared and visible image fusion10.1016/j.inffus.2018.09.004 ↗
21. FusionGRAM: An Infrared and Visible Image Fusion Framework Based on Gradient Residual and Attention Mechanism
22. Effective Pan-Sharpening With Transformer and Invertible Neural Network10.1109/TGRS.2022.3199210 ↗
23. Jia, X.Y., Zhu, C., Li, M.Z., Tang, W.Q., Zhou, W.L., and Soc, I.C. (2021, January 11–17). LLVIP: A Visible-infrared Paired Dataset for Low-light Vision. Proceedings of the 18th IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCVW54120.2021.00389 ↗
24. Progress in color night vision10.1117/1.OE.51.1.010901 ↗
25. Xu, H., Ma, J.Y., Le, Z.L., Jiang, J.J., Guo, X.J., and Assoc Advancement Artificial, I. (2020, January 7–12). FusionDN: A Unified Densely Connected Network for Image Fusion. Proceedings of the 34th AAAI Conference on Artificial Intelligence/32nd Innovative Applications of Artificial Intelligence Conference/10th AAAI Symposium on Educational Advances in Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i07.6936 ↗
26. MUFusion: A general unsupervised image fusion network based on memory unit10.1016/j.inffus.2022.11.010 ↗
27. Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network10.1016/j.inffus.2021.12.004 ↗
28. Cross-Modal Transformers for Infrared and Visible Image Fusion10.1109/TCSVT.2023.3289170 ↗
29. Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption10.1109/TPAMI.2024.3521416 ↗
30. Infrared and visible image fusion methods and applications: A survey10.1016/j.inffus.2018.02.004 ↗
31. Deep Learning-Based Multi-Focus Image Fusion: A Survey and a Comparative Study