journal article·2025-02-05Open access

Patch-Wise-Based Diffusion Model with Uncertainty Guidance for Low-Light Image Enhancement

Li Li, Jishen Peng, Yingcai Wan

Applied Sciences· Volume 15· Issue 3

Abstract

Low-light image enhancement is crucial for accurately interpreting images captured under low-lighting conditions. Existing low-light enhancement methods based on diffusion models have demonstrated effectiveness; however, they suffer from slow training processes and less structured guidance during optimization. We propose a navel patch-wise-based diffusion model, which introduces the low curvature reverse trajectory to ensure stable parameter optimization and uncertainty guidance in the diffusion training process. Specifically, we randomly select patches of varying sizes from the entire image and apply patch-wise optimization between the generated image and the ground truth to enforce a stable optimization path in the diffusion model. Additionally, within each patch-wise region, an uncertainty network estimates the uncertainty, which is then integrated as a weighting factor in the diffusion process to balance areas of abrupt change in the image. Experimental evaluations on various datasets demonstrate that our method achieves significant improvements, particularly in experiments with real-world images. These results indicate that the proposed patch-wise-based diffusion model enhancements are effective.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Image Enhancement Techniques · Image and Video Quality Assessment · Advanced Image Processing Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Faculty of Electrical and Control, Liaoning Technical University, Huludao 125000, China

Material Company, State Grid Xinjiang Electric Power Co., Ltd., Urumqi 830063, China

School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

References (76)

Show source-provided references
1. Banik, P.P., Saha, R., and Kim, K.-D. (2018, January 24–27). Contrast enhancement of low-light image using histogram equalization and illumination adjustment. Proceedings of the 2018 International Conference on Electronics, Information, and Communication (ICEIC), Honolulu, HI, USA.10.23919/ELINFOCOM.2018.8330564 ↗
2. Color enhancement of low exposure images using histogram specification and its application to color shift model-based refocusing
3. Histogram based perceptual quality assessment method for color images10.1016/j.csi.2014.04.002 ↗
4. Retinex image enhancement via a learned dictionary10.1117/1.OE.54.1.013107 ↗
5. Cai, Y., Bian, H., Lin, J., Wang, H., Timofte, R., and Zhang, Y. (2023, January 2–6). Retinexformer: One-stage retinex-based transformer for low-light image enhancement. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01149 ↗
6. Yi, X., Xu, H., Zhang, H., Tang, L., and Ma, J. (2023, January 2–6). Diff-retinex: Rethinking low-light image enhancement with a generative diffusion model. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01130 ↗
7. A reflectance re-weighted retinex model for non-uniform and low-light image enhancement10.1016/j.patcog.2023.109823 ↗
8. A fusion-based enhancing method for weakly illuminated images10.1016/j.sigpro.2016.05.031 ↗
9. Shen, L., Yue, Z., Feng, F., Chen, Q., Liu, S., and Ma, J. (2017). Msr-net: Low-light image enhancement using deep convolutional network. arXiv Prepr.
10. Lime: Low-light image enhancement via illumination map estimation10.1109/TIP.2016.2639450 ↗
11. Structure-revealing low-light image enhancement via robust retinex model10.1109/TIP.2018.2810539 ↗
12. Efinet: Restoration for low-light images via enhancement-fusion iterative network10.1109/TCSVT.2022.3195996 ↗
13. A mutually boosting dual sensor computational camera for high quality dark videography10.1016/j.inffus.2023.01.013 ↗
14. Treenhance: A tree search method for low-light image enhancement10.1016/j.patcog.2022.109249 ↗
15. Fan, M., Wang, W., Yang, W., and Liu, J. (2020, January 12–16). Integrating semantic segmentation and retinex model for low-light image enhancement. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.10.1145/3394171.3413757 ↗
16. Zhang, X., Shen, P., Luo, L., Zhang, L., and Song, J. (2012, January 11–15). Enhancement and noise reduction of very low light level images. Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), Tsukuba, Japan.
17. No-reference quality metric of contrast-distorted images based on information maximization10.1109/TCYB.2016.2575544 ↗
18. Ren, X., Li, M., Cheng, W.-H., and Liu, J. (2018, January 27–30). Joint enhancement and denoising method via sequential decomposition. Proceedings of the 2018 IEEE International Symposium on Circuits and Systems (ISCAS), Florence, Italy.10.1109/ISCAS.2018.8351427 ↗
19. Low-light image enhancement via progressive-recursive network10.1109/TCSVT.2021.3049940 ↗
20. Zhang, F., Li, Y., You, S., and Fu, Y. (2021, January 20–25). Learning temporal consistency for low light video enhancement from single images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00493 ↗
21. Wang, Y., Cao, Y., Zha, Z.-J., Zhang, J., Xiong, Z., Zhang, W., and Wu, F. (2019, January 21–25). Progressive retinex: Mutually reinforced illumination-noise perception network for low-light image enhancement. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.10.1145/3343031.3350983 ↗
22. Song, J., Meng, C., and Ermon, S. (2020). Denoising diffusion implicit models. arXiv.
23. Denoising diffusion restoration models
24. Restoring vision in adverse weather conditions with patch-based denoising diffusion models10.1109/TPAMI.2023.3238179 ↗
25. Low-light image enhancement with wavelet-based diffusion models
26. Wang, Y., Yu, Y., Yang, W., Guo, L., Chau, L.-P., Kot, A.C., and Wen, B. (2023, January 4–6). Exposurediffusion: Learning to expose for low-light image enhancement. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01143 ↗
27. Zhou, D., Yang, Z., and Yang, Y. (2023). Pyramid diffusion models for low-light image enhancement. arXiv Prepr.10.24963/ijcai.2023/199 ↗
28. Hou, J., Zhu, Z., Hou, J., Liu, H., Zeng, H., and Yuan, H. (2023, January 10–16). Global Structure-Aware Diffusion Process for Low-Light Image Enhancement. Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, USA.
29. Lightness and retinex theory10.1364/JOSA.61.000001 ↗
30. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
31. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗
32. Ding, X., Guo, Y., Ding, G., and Han, J. (November, January 27). Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks. Proceedings of the IEEE/CVF International Conference on Computer Vision.
33. Wang, R., Zhang, Q., Fu, C.-W., Shen, X., Zheng, W.-S., and Jia, J. (2019, January 15–20). Underexposed photo enhancement using deep illumination estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00701 ↗
34. Wang, J., Tan, W., Niu, X., and Yan, B. (2019, January 8–12). Rdgan: Retinex decomposition based adversarial learning for low-light enhancement. Proceedings of the 2019 IEEE International Conference on Multimedia and Expo (ICME), Shanghai, China.10.1109/ICME.2019.00207 ↗
35. Chen, C., Chen, Q., Xu, J., and Koltun, V. (2018, January 18–23). Learning to see in the dark. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00347 ↗
36. Dslr: Deep stacked laplacian restorer for low-light image enhancement10.1109/TMM.2020.3039361 ↗
37. Llnet: A deep autoencoder approach to natural low-light image enhancement10.1016/j.patcog.2016.06.008 ↗
38. Mbllen: Low-light image/video enhancement using cnns
39. Low-light image enhancement via a deep hybrid network10.1109/TIP.2019.2910412 ↗
40. Zhu, M., Pan, P., Chen, W., and Yang, Y. (2020, January 7–12). Eemefn: Low-light image enhancement via edge-enhanced multi-exposure fusion network. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA. No. 07.10.1609/aaai.v34i07.7013 ↗
41. A deep learning based image enhancement approach for autonomous driving at night10.1016/j.knosys.2020.106617 ↗
42. Xu, K., Yang, X., Yin, B., and Lau, R.W. (2020, January 14–19). Learning to restore low-light images via decomposition-and-enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00235 ↗
43. Low-light image enhancement with semi-decoupled decomposition10.1109/TMM.2020.2969790 ↗
44. Yang, W., Wang, S., Fang, Y., Wang, Y., and Liu, J. (2020, January 13–19). From fidelity to perceptual quality: A semi-supervised approach for low-light image enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00313 ↗
45. Retinexdip: A unified deep framework for low-light image enhancement10.1109/TCSVT.2021.3073371 ↗
46. DeepExposure: Learning to Expose Photos with Asynchronously Reinforced Adversarial Learning
47. Getting to know low-light images with the exclusively dark dataset10.1016/j.cviu.2018.10.010 ↗
48. Bychkovsky, V., Paris, S., Chan, E., and Durand, F. (2011, January 20–25). Learning photographic global tonal adjustment with a database of input/output image pairs. Proceedings of the CVPR, Colorado Springs, CO, USA.10.1109/CVPR.2011.5995332 ↗
49. Zhang, Y., Di, X., Zhang, B., and Wang, C. (2020). Self-supervised image enhancement network: Training with low light images only. arXiv.
50. Guo, C., Li, C., Guo, J., Loy, C.C., Hou, J., Kwong, S., and Cong, R. (2020, January 13–19). Zero-reference deep curve estimation for low-light image enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00185 ↗
51. Wu, Y., Wang, G., Wang, Z., Yang, Y., Li, T., Zhang, M., Li, C., and Shen, H.T. (2024). JoReS-Diff: Joint Retinex and Semantic Priors in Diffusion Model for Low-light Image Enhancement. arXiv, Available online: https://arxiv.org/abs/2312.12826.10.1145/3664647.3680876 ↗
52. Denoising Diffusion Probabilistic Models
53. Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., and Poole, B. (2020). Score-Based Generative Modeling through Stochastic Differential Equations. arXiv, Available online: https://arxiv.org/abs/2011.13456.
54. Ramesh, A., Pavlov, M., and Goh, G. (2021, January 10–17). Zero-Shot Text-to-Image Generation. Proceedings of the International Conference on Computer Vision (ICCV), Montreal, QC, Canada.
55. Hu, X., Liu, Y., and Wang, Z. (2023, January 18–22). Self-Supervised Signals for Diffusion Models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.
56. Boutin, A., Smith, B., and Doe, C. (2023, January 1–6). Diffusion Models for Generating Human-like Artistic Drawings. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.
57. Theoretical Analysis of Diffusion Models: Minimax Optimal Rates for Distribution Approximation
58. Wei, C., Wang, W., Yang, W., and Liu, J. (2018). Deep retinex decom position for low-light enhancement. arXiv Prepr.
59. Sparse gradient regularized deep retinex network for robust low-light image enhancement10.1109/TIP.2021.3050850 ↗
60. Lee, C., Lee, C., and Kim, C.-S. (October, January 30). Contrast enhancement based on layered difference representation. Proceedings of the 2012 19th IEEE International Conference on Image Processing, Lake Buena Vista, FL, USA.10.1109/ICIP.2012.6467022 ↗
61. Naturalness preserved enhancement algorithm for non-uniform illumination images10.1109/TIP.2013.2261309 ↗
62. Perceptual quality assessment for multi-exposure image fusion10.1109/TIP.2015.2442920 ↗
63. Vonikakis, V., Bouzos, O., and Andreadis, I. (2011, January 22–24). Multi-exposure image fusion based on illumination estimation. Proceedings of the IASTED International Conference on Signal and Image Processing and Applications (SIPA), Heraklion, Crete, Greece.10.2316/P.2011.738-051 ↗
64. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
65. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–23). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗
66. Making a ’completely blind’ image quality analyzer10.1109/LSP.2012.2227726 ↗
67. EnlightenGAN: Deep light enhancement without paired supervision10.1109/TIP.2021.3051462 ↗
68. Zhang, Y., Zhang, J., and Guo, X. (2019, January 21–25). Kindling the darkness: A practical low-light image enhancer. Proceedings of the 27th ACM International Conference on Multimedia (ACM MM 2019), Nice, France.10.1145/3343031.3350926 ↗
69. Beyond brightening low-light images10.1007/s11263-020-01407-x ↗
70. Learning enriched features for fast image restoration and enhancement10.1109/TPAMI.2022.3167175 ↗
71. Wang, T., Zhang, K., Shen, T., Luo, W., Stenger, B., and Lu, T. (2023, January 7–14). Ultra-high-definition low-light image enhancement: A benchmark and transformer-based method. Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI-23), Washington, DC, USA.10.1609/aaai.v37i3.25364 ↗
72. Wang, Y., Wan, R., Yang, W., Li, H., Chau, L., and Kot, A. (March, January 22). Low-light image enhancement with normalizing flow. Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI-22), Virtual Conference.10.1609/aaai.v36i3.20162 ↗
73. Xu, X., Wang, R., Fu, C., and Jia, J. (2022, January 18–24). SNR-aware low-light image enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01719 ↗
74. DCCNet: Low-light image enhancement via deep color correction network
75. Ma, L., Ma, T., Liu, R., Fan, X., and Luo, Z. (2022, January 18–24). Toward Fast, Flexible, and Robust Low-Light Image Enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00555 ↗
76. Lv, X., Zhang, S., Wang, C., Zheng, Y., Zhong, B., Li, C., and Nie, L. (2024, January 17–21). Fourier Priors-Guided Diffusion for Zero-Shot Joint Low-Light Enhancement and Deblurring. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.02398 ↗