sRrsR-Net: A New Low-Light Image Enhancement Network via Raw Image Reconstruction
Source abstract
Abstract
Most existing low-light image enhancement (LIE) methods are primarily designed for human-vision-friendly image formats, such as sRGB, due to their convenient storage and smaller file sizes. In addition, raw images provide greater detail and a wider dynamic range, which makes them more suitable for LIE tasks. Despite these advantages, raw images, the original format captured by cameras, are larger and less accessible and are hard to use in methods of LIE with mobile devices. In order to leverage both the advantages of sRGB and raw domains while avoiding the direct use of raw images as training data, this paper introduces sRrsR-Net, a novel framework with the image translation process of sRGB–raw–sRGB for LIE task. In our approach, firstly, the RGB-to-iRGB module is designed to convert sRGB images into intermediate RGB feature maps. Then, with these intermediate feature maps, to bridge the domain gap between sRGB and raw pixels, the RAWFormer module is proposed to employ global attention to effectively align features between the two domains to generate reconstructed raw images. For enhancing the raw images and restoring them back to normal-light sRGB, unlike traditional Image Signal Processing (ISP) pipelines, which are often bulky and integrate numerous processing steps, we propose the RRAW-to-sRGB module. This module simplifies the process by focusing only on color correction and white balance, while still delivering competitive results. Extensive experiments on four benchmark datasets referring to both domains demonstrate the effectiveness of our approach.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Image Enhancement Techniques · Advanced Image Processing Techniques · Image and Signal Denoising Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Electronic and Information Engineering, Wuyi University, Jiangmen 529000, China
Funding
Doctoral Research Initiation Project of Wuyi University
References (54)
Show source-provided references
1. Cui, Z., Li, K., Gu, L., Su, S., Gao, P., Jiang, Z., Qiao, Y., and Harada, T. (2022). You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correction. arXiv.
2. Tran, D.Q., Aboah, A., Jeon, Y., Shoman, M., Park, M., and Park, S. (2024, January 16–22). Low-Light Image Enhancement Framework for Improved Object Detection in Fisheye Lens Datasets. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPRW63382.2024.00699 ↗
3. Zhu, A., Zhang, L., Shen, Y., Ma, Y., Zhao, S., and Zhou, Y. (2020, January 6–10). Zero-Shot Restoration of Underexposed Images via Robust Retinex Decomposition. Proceedings of the 2020 IEEE International Conference on Multimedia and Expo (ICME), London, UK.10.1109/ICME46284.2020.9102962 ↗
4. Xu, X., Wang, R., Fu, C.-W., and Jia, J. (2022, January 18–24). SNR-Aware Low-Light Image Enhancement. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01719 ↗
5. Jiang, Y., Chang, S., and Wang, Z. (2021). TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up. arXiv.
6. Marnissi, M.A., and Fathallah, A. (2023, January 17–24). GAN-Based Vision Transformer for High-Quality Thermal Image Enhancement. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00089 ↗
7. Zhou, D., Yang, Z., and Yang, Y. (2023). Pyramid Diffusion Models For Low-Light Image Enhancement. arXiv.10.24963/ijcai.2023/199 ↗
8. Fei, B., Lyu, Z., Pan, L., Zhang, J., Yang, W., Luo, T., Zhang, B., and Dai, B. (2023, January 17–24). Generative Diffusion Prior for Unified Image Restoration and Enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00958 ↗
9. Cai, Y., Bian, H., Lin, J., Wang, H., Timofte, R., and Zhang, Y. (2023). Retinexformer: One-Stage Retinex-Based Transformer for Low-Light Image Enhancement. arXiv.10.1109/ICCV51070.2023.01149 ↗
10. Zhang, Z., Wang, H., Liu, M., Wang, R., Zhang, J., and Zuo, W. (2021, January 10–17). Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00431 ↗
11. Wang, Y., Yu, Y., Yang, W., Guo, L., Chau, L.-P., Kot, A., and Wen, B. (2023, January 17–24). Raw Image Reconstruction with Learned Compact Metadata. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.01746 ↗
12. Xu, S., Song, B., Chen, X., Liu, X., and Zhou, J. (2024). Image Demoireing in RAW and sRGB Domains. arXiv.10.1007/978-3-031-72658-3_7 ↗
13. Conde, M.V., Vasluianu, F., and Timofte, R. (2024, January 3–8). BSRAW: Improving Blind RAW Image Super-Resolution. Proceedings of the 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA.10.1109/WACV57701.2024.00831 ↗
14. Wei, K., Fu, Y., Yang, J., and Huang, H. (2020, January 9–13). A Physics-Based Noise Formation Model for Extreme Low-Light Raw Denoising. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00283 ↗
15. An Adaptive Gamma Correction for Image Enhancement10.1186/s13640-016-0138-1 ↗
16. Guo, C., Li, C., Guo, J., Loy, C.C., Hou, J., Kwong, S., and Cong, R. (2020, January 13–19). Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00185 ↗
17. Learning Enriched Features for Real Image Restoration and Enhancement10.1007/978-3-030-58595-2_30 ↗
18. Wang, R., Zhang, Q., Fu, C.-W., Shen, X., Zheng, W.-S., and Jia, J. (2019, January 15–20). Underexposed Photo Enhancement Using Deep Illumination Estimation. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00701 ↗
19. Li, J., Li, B., Tu, Z., Liu, X., Guo, Q., Juefei-Xu, F., Xu, R., and Yu, H. (2024, January 16–22). Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.01440 ↗
20. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., and Yang, M.-H. (2022, January 18–24). Restormer: Efficient Transformer for High-Resolution Image Restoration. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00564 ↗
21. Zhang, Z., Jiang, Y., Jiang, J., Wang, X., Luo, P., and Gu, J. (2021, January 10–17). STAR: A Structure-Aware Lightweight Transformer for Real-Time Image Enhancement. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00407 ↗
22. Towards Low Light Enhancement With RAW Images10.1109/TIP.2022.3140610 ↗
23. Chen, C., Chen, Q., Xu, J., and Koltun, V. (2018, January 18–22). Learning to See in the Dark. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00347 ↗
24. As-Projective-as-Possible Bias Correction for Illumination Estimation Algorithms10.1364/JOSAA.36.000071 ↗
25. Nam, S., Punnappurath, A., Brubaker, M.A., and Brown, M.S. (2022, January 18–24). Learning sRGB-to-Raw-RGB De-Rendering with Content-Aware Metadata. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01718 ↗
26. Debevec, P.E., and Malik, J. (2008, January 11). Recovering High Dynamic Range Radiance Maps from Photographs. Proceedings of the ACM SIGGRAPH 2008 Classes, Los Angeles, CA, USA.10.1145/1401132.1401174 ↗
27. Xing, Y., Qian, Z., and Chen, Q. (2021, January 20–25). Invertible Image Signal Processing. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00622 ↗
28. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.-H., and Shao, L. (2020, January 13–9). CycleISP: Real Image Restoration via Improved Data Synthesis. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00277 ↗
29. Boureau, Y.-L., Ponce, J., and Lecun, Y. (2010, January 21–24). A Theoretical Analysis of Feature Pooling in Visual Recognition. Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel.
30. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention Is All You Need. arXiv.
31. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., and Wang, Z. (2016, January 27–30). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.207 ↗
32. UniFormer: Unifying Convolution and Self-Attention for Visual Recognition10.1109/TPAMI.2023.3282631 ↗
33. Chollet, F. (2017, January 21–26). Xception: Deep Learning with Depthwise Separable Convolutions. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
34. Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
35. Wei, C., Wang, W., Yang, W., and Liu, J. (2018). Deep Retinex Decomposition for Low-Light Enhancement. arXiv.
36. Sparse Gradient Regularized Deep Retinex Network for Robust Low-Light Image Enhancement10.1109/TIP.2021.3050850 ↗
37. Benchmarking Low-Light Image Enhancement and Beyond10.1007/s11263-020-01418-8 ↗
38. Image Quality Assessment: From Error Visibility to Structural Similarity10.1109/TIP.2003.819861 ↗
39. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–23). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗
40. EnlightenGAN: Deep Light Enhancement Without Paired Supervision10.1109/TIP.2021.3051462 ↗
41. Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., and Li, H. (2022, January 18–4). Uformer: A General U-Shaped Transformer for Image Restoration. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01716 ↗
42. Liu, Y., Huang, T., Dong, W., Wu, F., Li, X., and Shi, G. (2023, January 1–6). Low-Light Image Enhancement with Multi-Stage Residue Quantization and Brightness-Aware Attention. Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France.10.1109/ICCV51070.2023.01115 ↗
43. Gu, S., Li, Y., Van Gool, L., and Timofte, R. (November, January 27). Self-Guided Network for Fast Image Denoising. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00260 ↗
44. Dong, X., Xu, W., Miao, Z., Ma, L., Zhang, C., Yang, J., Jin, Z., Teoh, A.B.J., and Shen, J. (2022, January 18–24). Abandoning the Bayer-Filter to See in the Dark. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01691 ↗
45. EEMEFN: Low-Light Image Enhancement via Edge-Enhanced Multi-Exposure Fusion Network10.1609/aaai.v34i07.7013 ↗
46. Maharjan, P., Li, L., Li, Z., Xu, N., Ma, C., and Li, Y. (2019, January 8–12). Improving Extreme Low-Light Image Denoising via Residual Learning. Proceedings of the 2019 IEEE International Conference on Multimedia and Expo (ICME), Shanghai, China.10.1109/ICME.2019.00162 ↗
47. Xu, K., Yang, X., Yin, B., and Lau, R.W.H. (2020, January 13–19). Learning to Restore Low-Light Images via Decomposition-and-Enhancement. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00235 ↗
48. Lamba, M., and Mitra, K. (2021, January 20–25). Restoring Extremely Dark Images in Real Time. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00349 ↗
49. LLNet: A Deep Autoencoder Approach to Natural Low-Light Image Enhancement10.1016/j.patcog.2016.06.008 ↗
50. Zhang, Y., Zhang, J., and Guo, X. (2019, January 15). Kindling the Darkness: A Practical Low-Light Image Enhancer. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.10.1145/3343031.3350926 ↗
51. Beyond Brightening Low-Light Images10.1007/s11263-020-01407-x ↗
52. Liu, R., Ma, L., Zhang, J., Fan, X., and Luo, Z. (2021, January 20–25). Retinex-Inspired Unrolling with Cooperative Prior Architecture Search for Low-Light Image Enhancement. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01042 ↗
53. Wang, Y., Wan, R., Yang, W., Li, H., Chau, L.-P., and Kot, A.C. (2021). Low-Light Image Enhancement with Normalizing Flow. arXiv.10.1609/aaai.v36i3.20162 ↗
54. Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., and Hassner, T. LEDNet: Joint Low-Light Enhancement and Deblurring in the Dark. Proceedings of the Computer Vision–ECCV 2022.10.1007/978-3-031-19836-6 ↗