Single Image Super-Resolution Method Using CNN-Based Lightweight Neural Networks
Source abstract
Abstract
There are many studies that seek to enhance a low resolution image to a high resolution image in the area of super-resolution. As deep learning technologies have recently shown impressive results on the image interpolation and restoration field, recent studies are focusing on convolutional neural network (CNN)-based super-resolution schemes to surpass the conventional pixel-wise interpolation methods. In this paper, we propose two lightweight neural networks with a hybrid residual and dense connection structure to improve the super-resolution performance. In order to design the proposed networks, we extracted training images from the DIVerse 2K (DIV2K) image dataset and investigated the trade-off between the quality enhancement performance and network complexity under the proposed methods. The experimental results show that the proposed methods can significantly reduce both the inference speed and the memory required to store parameters and intermediate feature maps, while maintaining similar image quality compared to the previous methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Processing Techniques · Advanced Vision and Imaging · Image and Signal Denoising Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Convergence IT Engineering, Kyungnam University, Changwon 51767, Korea
Department of Information and Communication Engineering, Kyungnam University, Changwon 51767, Korea
Department of IT Engineering, Sookmyung Women’s University, Seoul 04310, Korea
Intelligent Convergence Research Lab., Electronics and Telecommunications Research Institute (ETRI), Daejeon 34129, Korea
Funding
Electronics and Telecommunications Research Institute (ETRI)
References (26)
Show source-provided references
1. Deep Learning for Single Image Super-Resolution: A Brief Review10.1109/TMM.2019.2919431 ↗
2. Anwar, S., Khan, S., and Barnes, N. (2019). A deep journey into super-resolution: A survey. arXiv.
3. Wang, Z., Chen, J., and Hoi, S.C.H. (2020). Deep Learning for Image Super-resolution: A Survey. IEEE Trans. Pattern. Anal. Mach. Intell.10.1109/TPAMI.2020.2982166 ↗
4. Cubic convolution interpolation for digital image processing10.1109/TASSP.1981.1163711 ↗
5. Example-based super-resolution10.1109/38.988747 ↗
6. Efficient facial expression recognition algorithm based on hierarchical deep neural network structure10.1109/ACCESS.2019.2907327 ↗
7. deepGesture: Deep learning-based gesture recognition scheme using motion sensors10.1016/j.displa.2018.08.001 ↗
8. Jeong, D., Kim, B., and Dong, S. (2020). Deep Joint Spatiotemporal Network (DJSTN) for Efficient Facial Expression Recognition. Sensors, 20.10.3390/s20071936 ↗
9. Image super-resolution using deep convolutional networks10.1109/TPAMI.2015.2439281 ↗
10. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very deep convolution networks for large -scale image recognition. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA.
11. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the Conference on Computer Vision and Pattern Recognition, Las Vegas, NY, USA.10.1109/CVPR.2016.90 ↗
12. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
13. Ioffe, S., and Szegedy, C. (2015, January 7–9). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the Machine Learning Research, Lille, France.
14. Dong, C., Deng, Y., Change Loy, C., and Tang, X. (2015, January 7–13). Compression artifacts reduction by a deep convolutional network. Proceedings of the International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.73 ↗
15. Kim, J., Lee, J., and Lee, K. (2016, January 27–30). Accurate image super-resolution using very deep convolutional networks. Proceedings of the Conference on Computer Vision and Pattern Recognition, Las Vegas, NY, USA.10.1109/CVPR.2016.182 ↗
16. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21–26). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
17. Lim, B., Son, S., Kim, H., Nah, S., and Lee, K. (2017, January 21–26). Enhanced deep residual networks for single image super-resolution. Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.151 ↗
18. Tong, T., Li, G., Liu, X., and Gao, Q. (2017, January 21–26). Image super-resolution using dense skip connections. Proceedings of the Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/ICCV.2017.514 ↗
19. Ye, Y., Alshina, E., Chen, J., Liu, S., Pfaff, J., and Wang, S. ([DNNVC] AhG on Deep neural networks based video coding, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC29, Document JVET-T0121, 2020). [DNNVC] AhG on Deep neural networks based video coding, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC29, Document JVET-T0121, Input Document to JVET Teleconference.
20. Cho, S., Lee, J., Kim, J., Kim, Y., Kim, D., Chung, J., and Jung, S. (2019, January 16–20). Low Bit-rate Image Compression based on Post-processing with Grouped Residual Dense Network. Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.
21. Enhancing quality for HEVC compressed videos10.1109/TCSVT.2018.2867568 ↗
22. Recursive Residual Convolutional Neural Network-Based In-Loop Filtering for Intra Frames
23. Lui, G., Shih, K., Wang, T., Reda, F., Sapra, K., Yu, Z., Tao, A., and Catanzaro, B. (2018). Partial convolution based padding. arXiv.
24. Agustsson, E., and Timofte, R. (2017, January 21–26). NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.150 ↗
25. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
26. Yang, C., Ma, C., and Yang, M. (2014, January 6–12). Single-image super-resolution: A benchmark. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10593-2_25 ↗