Single Image Super-Resolution Method Based on an Improved Adversarial Generation Network
Source abstract
Abstract
Super-Resolution (SR) techniques for image restoration have recently been gaining attention due to their excellent performance. For powerful learning abilities, Generative Adversarial Networks (GANs) have been proven to have achieved great success. In this paper, we propose an Enhanced Generative Adversarial Network (EGAN) for improving its effects for a real-time Super-Resolution task. The main content of this paper are as follows: (1) We adopted the Laplacian pyramid framework as a pre-trained module, which is beneficial for providing multiscale features for our input. (2) At each feature block, a convolutional skip-connections network, which may contain some latent information, was significant for the generative model to reconstruct a plausible-looking image. (3) Considering that the edge details usually play an important role in image generation, a perceptual loss function was defined to train and seek the optimal parameters. Quantitative and qualitative evaluations were demonstrated so that our algorithm not only took full advantage of the Convolutional Neural Networks (CNNs) to improve the image quality, but also performed better than other algorithms in speed and performance for real-time Super-Resolution tasks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Processing Techniques · Image Processing Techniques and Applications · Advanced Vision and Imaging
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Equipment Management and Unmanned Aerial Vehicle Engineering School, Air Force Engineering University, Xi’an 710043, China
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China
References (40)
Show source-provided references
1. Dong, J., Hong, Z., Yuan, D., Chen, H., and You, Y. (2015, January 23–25). Fast video super-resolution via sparse coding. Proceedings of the International Conference on Graphic & Image Processing, Singapore.10.1117/12.2179397 ↗
2. Image Super-Resolution Using Deep Convolutional Networks10.1109/TPAMI.2015.2439281 ↗
3. Chao, D., Chen, C.L., He, K., and Tang, X. (2014, January 6–12). Learning a Deep Convolutional Network for Image Super-Resolution. Proceedings of the European Conference on Computer Vision (ECCV), Zurich, Switzerland.
4. Kim, J., Lee, J.K., and Lee, K.M. (July, January 26). Accurate Image Super-Resolution Using Very Deep Convolutional Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
5. Lai, W.S., Huang, J.B., Ahuja, N., and Yang, M.H. (2017, January 21–26). Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.618 ↗
6. Ying, T., Jian, Y., and Liu, X. (2017, January 21–26). Image Super-Resolution via Deep Recursive Residual Network. Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition, Honolulu, HI, USA.
7. How Generative Adversarial Nets and its variants Work: An Overview of GAN
8. Doersch, C. (2016). Tutorial on Variational Autoencoders. arXiv.
9. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Nets, MIT Press.
10. Lucic, M., Kurach, K., Michalski, M., Gelly, S., and Bousquet, O. (2017). Are GANs Created Equal? A Large-Scale Study. arXiv.
11. Linear interpolation revitalized10.1109/TIP.2004.826093 ↗
12. Timofte, R., Desmet, V., and Vangool, L. (2014, January 1–5). A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution. Proceedings of the Asian Conference on Computer Vision, Singapore.10.1007/978-3-319-16817-3_8 ↗
13. Wang, Z., Liu, D., Yang, J., Han, W., and Huang, T. (2016, January 27–30). Deep Networks for Image Super-Resolution with Sparse Prior. Proceedings of the IEEE International Conference on Computer Vision, Las Vegas, NV, USA.10.1109/ICCV.2015.50 ↗
14. Agustsson, E., and Timofte, R. (2017, January 21–26). NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA.10.1109/CVPRW.2017.150 ↗
15. Shi, W., Caballero, J., Huszár, F., Totz, J., and Wang, Z. (2016, January 27–30). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.207 ↗
16. Exploring Context with Deep Structured models for Semantic Segmentation10.1109/TPAMI.2017.2708714 ↗
17. Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv.
18. Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (July, January 26). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
19. Kim, J., Lee, J.K., and Lee, K.M. (July, January 26). Deeply-Recursive Convolutional Network for Image Super-Resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
20. Denton, E., Chintala, S., Szlam, A., and Fergus, R. (2015, January 8–12). Deep Generative Image Models Using a Laplacian Pyramid of Adversarial Networks. Proceedings of the 28th International Conference on Neural Information Processing Systems, Sanur, Bali, Indonesia.
21. Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., and Abbeel, P. (2016). InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. arXiv.
22. Mirza, M., and Osindero, S. (2014). Conditional Generative Adversarial Nets. arXiv.
23. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
24. Zhang, K., Gao, X., Tao, D., and Li, X. (2012, January 16–21). Multi-scale dictionary for single image super-resolution. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.
25. Yang, J., Wright, J., Huang, T.S., and Yi, M. (2008, January 24–26). Image super-resolution as sparse representation of raw image patches. Proceedings of the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, AK, USA.
26. Zeyde, R., Elad, M., and Protter, M. (2010). On Single Image Scale-Up Using Sparse-Representations, Springer.
27. Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv.
28. Lin, G., Milan, A., Shen, C., and Reid, I. (2017, January 21–26). RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.549 ↗
29. He, K., Zhang, X., Ren, S., and Jian, S. (2016). Identity Mappings in Deep Residual Networks, Springer.10.1007/978-3-319-46493-0_38 ↗
30. Mao, X., Li, Q., Xie, H., Lau, R., and Smolley, S.P. (2017, January 22–29). Least Squares Generative Adversarial Networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.304 ↗
31. Gross, S., and Wilber, M. (2022, June 09). Training and Investigating Residual Nets. Torch, Available online: http://torch.ch/blog/2016/02/04/resnets.html.
32. Ioffe, S., and Szegedy, C. (2015, January 6–11). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the International Conference on Machine Learning, Lille, France.
33. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A. (2017, January 4–9). Improved Training of Wasserstein GANs. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
34. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs10.1109/TPAMI.2017.2699184 ↗
35. Bevilacqua, M., Roumy, A., Guillemot, C., and Morel, A. (2012, January 3–7). Low-Complexity Single Image Super-Resolution Based on Nonnegative Neighbor Embedding. Proceedings of the 23rd British Machine Vision Conference (BMVC), Surrey, UK.10.5244/C.26.135 ↗
36. Martin, D., Fowlkes, C., Tal, D., and Malik, J. (2001, January 7–14). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of the Eighth IEEE International Conference on Computer Vision, Vancouver, BC, Canada.
37. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
38. Zeyde, R., Elad, M., and Protter, M. (2010, January 24–30). On single image scale-up using sparse-representations. Proceedings of the International Conference on Curves and Surfaces, Avignon, France.
39. Huang, J.B., Singh, A., and Ahuja, N. (2015, January 7–12). Single image super-resolution from transformed self-exemplars. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7299156 ↗
40. Kingma, D., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv.