journal article·2023-01-07Open access

GPSR: Gradient-Prior-Based Network for Image Super-Resolution

Xiancheng Zhu, Detian Huang, Xiaorui Li, Danlin Cai, Daxin Zhu

Applied Sciences· Volume 13· Issue 2

Abstract

Recent deep learning has shown great potential in super-resolution (SR) tasks. However, most deep learning-based SR networks are optimized via pixel-level loss (i.e., L1, L2, and MSE), which forces the networks to output the average of all possible predictions, leading to blurred details. Especially in SR tasks with large scaling factors (i.e., ×4, ×8), the limitation is further aggravated. To alleviate this limitation, we propose a Gradient-Prior-based Super-Resolution network (GPSR). Specifically, a detail-preserving Gradient Guidance Strategy is proposed to fully exploit the gradient prior to guide the SR process from two aspects. On the one hand, an additional gradient branch is introduced into GPSR to provide the critical structural information. On the other hand, a compact gradient-guided loss is proposed to strengthen the constraints on the spatial structure and to prevent the blind restoration of high-frequency details. Moreover, two residual spatial attention adaptive aggregation modules are proposed and incorporated into the SR branch and the gradient branch, respectively, to fully exploit the crucial intermediate features to enhance the feature representation ability. Comprehensive experimental results demonstrate that the proposed GPSR outperforms state-of-the-art methods regarding both subjective visual quality and objective quantitative metrics in SR tasks with large scaling factors (i.e., ×4 and ×8).
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Image Processing Techniques · Advanced Vision and Imaging · Image Processing Techniques and Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

College of Engineering, Huaqiao University, Quanzhou 362021, China

College of Fine Arts, Huaqiao University, Quanzhou 362021, China

School of Mathematics and Computer Science, Quanzhou Normal University, Quanzhou 362021, China

Funding

National Natural Science Foundation of China · National Key R&D Program of China · Collaborative Innovation Platform Project of Fujian Province · Natural Science Foundation of Fujian Provincial Science and Technology Department · Natural Science Foundation of Fujian Province · Fundamental Research Funds for the Central Universities · Foundation of Fujian Education Department · Key Project of Quanzhou Science and Technology Plan

References (46)

Show source-provided references
1. Arbitrary-oriented object detection via dense feature fusion and attention model for remote sensing super-resolution image10.1007/s00521-020-04893-9 ↗
2. Super-resolution for remote sensing images via local–global combined network10.1109/LGRS.2017.2704122 ↗
3. Superresolution in MRI: Application to human white matter fiber tract visualization by diffusion tensor imaging10.1002/1522-2594(200101)45:1<29::AID-MRM1005>3.0.CO;2-Z ↗
4. Sub-pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super-resolution pixel-swapping10.1080/01431160500207088 ↗
5. Chen, Y., Tai, Y., Liu, X., Shen, C., and Yang, J. (2018, January 18–22). Fsrnet: End-to-end learning face super-resolution with facial priors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00264 ↗
6. Yang, T., Ren, P., Xie, X., and Zhang, L. (2021, January 20–25). GAN prior embedded network for blind face restoration in the wild. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00073 ↗
7. Very low resolution face recognition problem10.1109/TIP.2011.2162423 ↗
8. A super-resolution reconstruction algorithm for surveillance images10.1016/j.sigpro.2009.09.002 ↗
9. Dong, C., Loy, C.C., He, K., and Tang, X. (2014, January 6–12). Learning a deep convolutional network for image super-resolution. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10593-2_13 ↗
10. Lim, B., Son, S., Kim, H., Nah, S., and Mu Lee, K. (2017, January 21–26). Enhanced deep residual networks for single image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.151 ↗
11. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., and Fu, Y. (2018, January 18–22). Residual dense network for image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00262 ↗
12. Niu, B., Wen, W., Ren, W., Zhang, X., Yang, L., Wang, S., Zhang, K., Cao, X., and Shen, H. (2020, January 23–28). Single image super-resolution via a holistic attention network. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58610-2_12 ↗
13. Mei, Y., Fan, Y., and Zhou, Y. (2021, January 20–25). Image super-resolution with non-local sparse attention. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00352 ↗
14. Mei, Y., Fan, Y., Zhang, Y., Yu, J., Zhou, Y., Liu, D., Fu, Y., Huang, T.S., and Shi, H. (2020). Pyramid attention networks for image restoration. arXiv.
15. Dai, T., Cai, J., Zhang, Y., Xia, S.T., and Zhang, L. (2019, January 15–20). Second-order attention network for single image super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.01132 ↗
16. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint10.1002/cpa.20042 ↗
17. Computational methods for sparse solution of linear inverse problems10.1109/JPROC.2010.2044010 ↗
18. Image super-resolution by TV-regularization and Bregman iteration10.1007/s10915-008-9214-8 ↗
19. Image up-sampling using total-variation regularization with a new observation model10.1109/TIP.2005.851684 ↗
20. Peyré, G., Bougleux, S., and Cohen, L. (2008, January 12–18). Non-local regularization of inverse problems. Proceedings of the European Conference on Computer Vision, Marseille, France.10.1007/978-3-540-88690-7_5 ↗
21. Ma, C., Rao, Y., Cheng, Y., Chen, C., Lu, J., and Zhou, J. (2020, January 14–19). Structure-preserving super resolution with gradient guidance. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00779 ↗
22. Deep edge guided recurrent residual learning for image super-resolution10.1109/TIP.2017.2750403 ↗
23. Wei, P., Xie, Z., Lu, H., Zhan, Z., Ye, Q., Zuo, W., and Lin, L. (2020, January 23–28). Component divide-and-conquer for real-world image super-resolution. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58598-3_7 ↗
24. Kim, J., Lee, J.K., and Lee, K.M. (2016, January 27–30). Accurate Image Super-Resolution Using Very Deep Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.182 ↗
25. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
26. Liu, J., Zhang, W., Tang, Y., Tang, J., and Wu, G. (2020, January 13–19). Residual feature aggregation network for image super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00243 ↗
27. Li, Z., Yang, J., Liu, Z., Yang, X., Jeon, G., and Wu, W. (2019, January 15–20). Feedback network for image super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00399 ↗
28. A model of saliency-based visual attention for rapid scene analysis10.1109/34.730558 ↗
29. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., and Fu, Y. (2018, January 8–14). Image super-resolution using very deep residual channel attention networks. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_18 ↗
30. Mei, Y., Fan, Y., Zhou, Y., Huang, L., Huang, T.S., and Shi, H. (2020, January 13–19). Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00573 ↗
31. Channel-wise and spatial feature modulation network for single image super-resolution10.1109/TCSVT.2019.2915238 ↗
32. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
33. Wang, X., Girshick, R., Gupta, A., and He, K. (2018, January 18–22). Non-local neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00813 ↗
34. Haris, M., Shakhnarovich, G., and Ukita, N. (2018, January 18–22). Deep back-projection networks for super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00179 ↗
35. Liu, Y., Wang, Y., Li, N., Cheng, X., Zhang, Y., Huang, Y., and Lu, G. (2018, January 20–24). An attention-based approach for single image super resolution. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.10.1109/ICPR.2018.8545760 ↗
36. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., and Wang, Z. (2016, January 27–30). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.207 ↗
37. Wang, Y., Perazzi, F., McWilliams, B., Sorkine-Hornung, A., Sorkine-Hornung, O., and Schroers, C. (2018, January 18–22). A fully progressive approach to single-image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.10.1109/CVPRW.2018.00131 ↗
38. Lai, W.S., Huang, J.B., Ahuja, N., and Yang, M.H. (2017, January 21–26). Deep laplacian pyramid networks for fast and accurate super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.618 ↗
39. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21–26). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
40. Guo, Y., Chen, J., Wang, J., Chen, Q., Cao, J., Deng, Z., Xu, Y., and Tan, M. (2020, January 13–19). Closed-loop matters: Dual regression networks for single image super-resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00545 ↗
41. Timofte, R., Agustsson, E., Van Gool, L., Yang, M.H., and Zhang, L. (2017, January 21–26). Ntire 2017 challenge on single image super-resolution: Methods and results. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.150 ↗
42. Bevilacqua, M., Roumy, A., Guillemot, C., and Alberi-Morel, M.L. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. Proceedings of the British Machine Vision Conference.
43. Zeyde, R., Elad, M., and Protter, M. (2010, January 24–30). On single image scale-up using sparse-representations. Proceedings of the International Conference on Curves and Surfaces, Avignon, France.
44. Martin, D., Fowlkes, C., Tal, D., and Malik, J. (2001, January 7–14). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of the Eighth IEEE International Conference on Computer Vision, ICCV 2001, Vancouver, BC, Canada.
45. Huang, J.B., and Singh, A. (2015, January 7–12). Single image super-resolution from transformed self-exemplars. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7299156 ↗
46. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.