A Method for Style Transfer from Artistic Images Based on Depth Extraction Generative Adversarial Network
Source abstract
Abstract
Depth extraction generative adversarial network (DE-GAN) is designed for artistic work style transfer. Traditional style transfer models focus on extracting texture features and color features from style images through an autoencoding network by mixing texture features and color features using high-dimensional coding. In the aesthetics of artworks, the color, texture, shape, and spatial features of the artistic object together constitute the artistic style of the work. In this paper, we propose a multi-feature extractor to extract color features, texture features, depth features, and shape masks from style images with U-net, multi-factor extractor, fast Fourier transform, and MiDas depth estimation network. At the same time, a self-encoder structure is used as the content extraction network core to generate a network that shares style parameters with the feature extraction network and finally realizes the generation of artwork images in three-dimensional artistic styles. The experimental analysis shows that compared with other advanced methods, DE-GAN-generated images have higher subjective image quality, and the generated style pictures are more consistent with the aesthetic characteristics of real works of art. The quantitative data analysis shows that images generated using the DE-GAN method have better performance in terms of structural features, image distortion, image clarity, and texture details.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Aesthetic Perception and Analysis · Computer Graphics and Visualization Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Virtual Reality Research Institute, Nanchang Hangkong University, Nanchang 330063, China
Jiangxi Institute of Science and Technology Information, Nanchang 330046, China
Funding
General projects of Humanities and Social Sciences in colleges and universities of Jiangxi province · General Art Planning Project of Jiangxi Province
References (24)
Show source-provided references
1. Efros, A.A., and Freeman, W.T. (2001, January 12–17). Image quilting for texture synthesis and transfer. Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA.10.1145/383259.383296 ↗
2. Efros, A.A., and Leung, T.K. (1999, January 20–27). Texture synthesis by non-parametric sampling. Proceedings of the Seventh IEEE International Conference on Computer Vision, Corfu, Greece.10.1109/ICCV.1999.790383 ↗
3. Gatys, L.A., Ecker, A.S., and Bethge, M. (2016, January 27–30). Image style transfer using convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.265 ↗
4. Imagenet classification with deep convolutional neural networks10.1145/3065386 ↗
5. Deep learning10.1038/nature14539 ↗
6. Universal style transfer via feature transforms
7. Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.244 ↗
8. Benaim, S., and Wolf, L. (2017, January 4–9). One-sided unsupervised domain mapping. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
9. Yu, Y., Gong, Z., Zhong, P., and Shan, J. (2017, January 13–15). Unsupervised representation learning with deep convolutional neural network for remote sensing images. Proceedings of the International Conference on Image and Graphics, Shanghai, China.10.1007/978-3-319-71589-6_9 ↗
10. A paired sparse representation model for robust face recognition from a single sample10.1016/j.patcog.2019.107129 ↗
11. Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion10.1016/j.inffus.2020.04.006 ↗
12. Chen, Y., Lai, Y.K., and Liu, Y.J. (2018, January 18–23). Cartoongan: Generative adversarial networks for photo cartoonization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00986 ↗
13. He, B., Gao, F., Ma, D., Shi, B., and Duan, L.Y. (2018, January 22–26). Chipgan: A generative adversarial network for chinese ink wash painting style transfer. Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Republic of Korea.10.1145/3240508.3240655 ↗
14. Karras, T., Laine, S., and Aila, T. (2019, January 15–20). A style-based generator architecture for generative adversarial networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00453 ↗
15. Upchurch, P., Gardner, J., Pleiss, G., Pless, R., Snavely, N., Bala, K., and Weinberger, K. (2017, January 21–26). Deep feature interpolation for image content changes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.645 ↗
16. Li, Y., Huang, C., and Loy, C.C. (2019, January 15–20). Dense intrinsic appearance flow for human pose transfer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00381 ↗
17. GANILLA: Generative adversarial networks for image to illustration translation10.1016/j.imavis.2020.103886 ↗
18. Ge, Y., Xiao, Y., Xu, Z., Wang, X., and Itti, L. (2022, January 23–24). Contributions of Shape, Texture, and Color in Visual Recognition. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-19775-8_22 ↗
19. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
20. Texture synthesis using convolutional neural networks
21. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer10.1109/TPAMI.2020.3019967 ↗
22. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2016, January 27–30). Learning deep features for discriminative localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.319 ↗
23. Image quality assessment through FSIM, SSIM, MSE and PSNR—A comparative study10.4236/jcc.2019.73002 ↗
24. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗