Generating Cartoon Images from Face Photos with Cycle-Consistent Adversarial Networks
Source abstract
Abstract
The generative adversarial network (GAN) is first proposed in 2014, and this kind of network model is machine learning systems that can learn to measure a given distribution of data, one of the most important applications is style transfer. Style transfer is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image. CYCLE-GAN is a classic GAN model, which has a wide range of scenarios in style transfer. Considering its unsupervised learning characteristics, the mapping is easy to be learned between an input image and an output image. However, it is difficult for CYCLE-GAN to converge and generate high-quality images. In order to solve this problem, spectral normalization is introduced into each convolutional kernel of the discriminator. Every convolutional kernel reaches Lipschitz stability constraint with adding spectral normalization and the value of the convolutional kernel is limited to [0, 1], which promotes the training process of the proposed model. Besides, we use pretrained model (VGG16) to control the loss of image content in the position of l1 regularization. To avoid overfitting, l1 regularization term and l2 regularization term are both used in the object loss function. In terms of Frechet Inception Distance (FID) score evaluation, our proposed model achieves outstanding performance and preserves more discriminative features. Experimental results show that the proposed model converges faster and achieves better FID scores than the state of the art.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Face recognition and analysis · Advanced Image Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (59)
Show source-provided references
1. Generative adversarial nets
2. A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow and B. Frey, “Adversarial autoencoders,” 2015. [Online]. Available: https://arxiv.org/abs/1511.05644.
3. J. Kim, M. Kim, K. Hyeonwoo and H. L. Kwang, “U-GAT-IT: Unsupervised generative attentional networks with adaptive layer-Instance normalization for image-to-image translation,” 2020. [Online]. Available: https://arxiv.org/abs/1907.10830.
4. M. Arjovsky, S. Chintala and L. Bottou, “Wasserstein GAN,” 2017. [Online]. Available: https://arxiv.org/abs/1701.07875.
5. Improved training of wasserstein GANs
6. J. B. Zhao, M. Mathieu and Y. LeCun, “Energy-based generative adversarial networks,” 2017. [Online]. Available: https://arxiv.org/abs/1609.03126.
7. X. B. Peng, A. Kanazawa, S. Toyer, P. Abbeel and S. Levine, “Variational discriminator bottleneck: Improving imitation learning, inverse RL, and GANs by constraining information flow,” 2018. [Online]. Available: https://arxiv.org/abs/1810.00821.
8. A. Radford, L. Metz and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” 2016. [Online]. Available: https://arxiv.org/abs/1511.06434.
9. V. Dumoulin, J. Shlens and M. Kudlur, “A learned representation for artistic style,” 2016. [Online]. Available: https://arxiv.org/abs/1610.07629.
10. Analyzing and improving the image quality of StyleGAN
11. Real-time monocular depth estimation using synthetic data with domain adaptation via Image style transfer
12. Image-to-image translation with conditional adversarial networks
13. Coupled generative adversarial networks
14. Unpaired image-to-image translation using cycle-consistent adversarial networks
15. StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation
16. StarGAN v2: Diverse image synthesis for multiple domains
17. CartoonGAN: Generative adversarial networks for photo cartoonization
18. T. Kim, C. Moonsu, H. Kim, J. K. Lee and J. Kim, “Learning to discover cross-domain relations with generative adversarial networks,” 2019. [Online]. Available: https://arxiv.org/abs/1703.05192.
19. DualGAN: Unsupervised dual learning for image-to-image translation
20. AttGAN: Facial attribute editing by only changing what you want10.1109/TIP.2019.2916751 ↗
21. PSGAN: Pose and expression robust spatial-aware GAN for customizable makeup transfer
22. Image style transfer using convolutional neural networks
23. A style-based generator architecture for generative adversarial networks
24. Image2StyleGAN: How to embed images into the StyleGAN latent space
25. J. Gui, Z. N. Sun, Y. G. Wen, D. C. Tao and J. P. Ye, “A review on generative adversarial networks: Algorithms, theory and applications,” 2020. [Online]. Available: https://arxiv.org/abs/2001.06937.
26. A. Brock, J. Donahue and K. Simonyan, “Large scale GAN training for high fidelity natural image synthesis,” 2018. [Online]. Available: https://arxiv.org/abs/1809.11096.
27. T. Karras, T. Aila, S. Laine and J. Lehtinen, “Progressive growing of GANs for improved quality, stability, and variation,” 2018. [Online]. Available: https://arxiv.org/abs/1710.10196.
28. EfficientNet: Rethinking model scaling for convolutional neural networks
29. MoCoGAN: Decomposing motion and content for video generation
30. Fashion editing with adversarial parsing learning
31. T. Miyato, T. Kataoka, M. Koyama and Y. Yoshida, “Spectral norm regularization for improving the generalizability of deep learning,” 2017. [Online]. Available: https://arxiv.org/abs/1705.10941.
32. D. Ulyanov, A. Vedaldi and V. S. Lempitsky, “Instance normalization: The missing ingredient for fast stylization,” 2016. [Online]. Available: https://arxiv.org/abs/1607.08022.
33. T. Miyato, T. Kataloka, M. Koyama and Y. Yoshida, “Spectral normalization for generative adversarial networks,” 2018. [Online]. Available: https://arxiv.org/abs/1802.05957.
34. Batch renormalization: Towards reducing minibatch dependence in batch-normalized models
35. Group normalization
36. Arbitrary style transfer in real-time with adaptive instance normalization
37. Batch normalization: Accelerating deep network training by reducing internal covariate shift
38. Weight normalization: A simple reparameterization to accelerate training of deep neural networks
39. Loss-sensitive generative adversarial networks on Lipschitz densities10.1007/s11263-019-01265-2 ↗
40. CCNet: criss-cross attention for semantic segmentation
41. Your local GAN: Designing two dimensional local attention mechanisms for generative models
42. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014. [Online]. Available: https://arxiv.org/abs/1409.1556.
43. Stabilizing training of generative adversarial networks through regularization
44. Which training methods for GANs do actually converge
45. A neural algorithm of artistic style10.1167/16.12.326 ↗
46. E. Risser, W. Pierre and B. Connelly, “Stable and controllable neural texture synthesis and style transfer using histogram losses,” 2017. [Online]. Available: https://arxiv.org/abs/1701.08893.
47. R. Yin, “Content aware neural style transfer,” 2016. [Online]. Available: https://arxiv.org/abs/1601.04568.
48. Combining Markov random fields and convolutional neural networks for image synthesis
49. Visual attribute transfer through deep image analogy
50. Perceptual losses for real-time style transfer and super-resolution
51. Real-time neural style transfer for videos
52. Texture networks: Feed-forward synthesis of textures and stylized images
53. Universal style transfer via feature transforms
54. Grad-CAM: Visual explanations from deep networks via gradient-based localization10.1007/s11263-019-01228-7 ↗
55. Robust design of deep neural networks against adversarial attacks based on lyapunov theory
56. Deep residual learning for image recognition
57. Learning representations by back-propagating errors
58. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2015. [Online]. Available: https://arxiv.org/abs/1412.6980.
59. GANs trained by a two time-scale update rule converge to a local nash equilibrium