Express Construction for GANs from Latent Representation to Data Distribution
Source abstract
Abstract
Generative Adversarial Networks (GANs) are powerful generative models for numerous tasks and datasets. However, most of the existing models suffer from mode collapse. The most recent research indicates that the reason for it is that the optimal transportation map from random noise to the data distribution is discontinuous, but deep neural networks (DNNs) can only approximate continuous ones. Instead, the latent representation is a better raw material used to construct a transportation map point to the data distribution than random noise. Because it is a low-dimensional mapping related to the data distribution, the construction procedure seems more like expansion rather than starting all over. Besides, we can also search for more transportation maps in this way with smoother transformation. Thus, we have proposed a new training methodology for GANs in this paper to search for more transportation maps and speed the training up, named Express Construction. The key idea is to train GANs with two independent phases for successively yielding latent representation and data distribution. To this end, an Auto-Encoder is trained to map the real data into the latent space, and two couples of generators and discriminators are used to produce them. To the best of our knowledge, we are the first to decompose the training procedure of GAN models into two more uncomplicated phases, thus tackling the mode collapse problem without much more computational cost. We also provide theoretical steps toward understanding the training dynamics of this procedure and prove assumptions. No extra hyper-parameters have been used in the proposed method, which indicates that Express Construction can be used to train any GAN models. Extensive experiments are conducted to verify the performance of realistic image generation and the resistance to mode collapse. The results show that the proposed method is lightweight, effective, and less prone to mode collapse.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Music and Audio Processing · Digital Media Forensic Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
Funding
the Science and Technology Achievements Transformation Demonstration Project of Sichuan Province of China · the Fundamental Research Funds for the Central Universities
References (36)
Show source-provided references
1. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8–13). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, QC, Canada.
2. Aaron van den Oord, N.K., and Kavukcuoglu, K. (2016, January 19–24). Pixel Recurrent Neural Networks. Proceedings of the International Conference on Machine Learning (ICML), New York, NY, USA.
3. Van den Oord, A., Kalchbrenner, N., Espeholt, L., Vinyals, O., and Graves, A. (2016, January 5–10). Conditional Image Generation with PixelCNN Decoders. Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, Barcelona, Spain.
4. Kingma, D., and Max, W. (2014, January 14–16). Auto-Encoding Variational Bayes. Proceedings of the International Conference on Learning Representations (ICLR), Banff, AB, Canada.
5. Tolstikhin Ilya, B.O., and Sylvain, G. (May, January 30). Wasserstein Auto-Encoders. Proceedings of the International Conference on Learning Representations(ICLR), Vancouver, BC, Canada.
6. Guo, Y., An, D., Qi, X., Luo, Z., Yau, S.T., and Gu, X. (2019). Mode Collapse and Regularity of Optimal Transportation Maps. arXiv.
7. An, D., Guo, Y., Lei, N., Luo, Z., Yau, S.T., and Gu, X. (2020, January 26–30). AE-OT: A New Generative Model Based on Extended Semi-discrete Optimal Transport. Proceedings of the International Conference on Learning Representations (ICLR), Addis Ababa, Ethiopia.
8. Nagarajan, V., and Kolter, J.Z. (2017). Gradient Descent GAN Optimization Is Locally Stable. Advances in Neural Information Processing Systems (NIPS), MIT Press.
9. A global geometric framework for nonlinear dimensionality reduction10.1126/science.290.5500.2319 ↗
10. Modeling word perception using the Elman network10.1016/j.neucom.2008.04.030 ↗
11. Hoang, Q., Nguyen, T.D., Le, T., and Phung, D. (May, January 30). MGAN: Training Generative Adversarial Nets with Multiple Generators. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada.
12. MADGAN: Unsupervised medical anomaly detection GAN using multiple adjacent brain MRI slice reconstruction10.1186/s12859-020-03936-1 ↗
13. Metz, L., Poole, B., Pfau, D., and Sohl-Dickstein, J. (2017, January 24–26). Unrolled Generative Adversarial Networks. Proceedings of the International Conference on Learning Representations (ICLR), Toulon, France.
14. Liu, H., Guo, Y., Lei, N., Shu, Z., Yau, S.T., Samaras, D., and Gu, X. (2018). Latent Space Optimal Transport for Generative Models. arXiv.
15. An, D., Guo, Y., Zhang, M., Qi, X., Lei, N., and Gu, X. (2020, January 23–28). AE-OT-GAN: Training GANs from Data Specific Latent Distribution. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.10.1007/978-3-030-58574-7_33 ↗
16. Arjovsky, M., and Bottou, L. (2017, January 24–26). Towards Principled Methods for Training Generative Adversarial Networks. Proceedings of the International Conference on Learning Representations (ICLR), Toulon, France.
17. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
18. Lin, Z., Khetan, A., Fanti, G., and Oh, S. (2018). PacGAN: The power of two samples in generative adversarial networks. Advances in Neural Information Processing Systems (NIPS), MIT Press.
19. Krizhevsky, A., and Hinton, G.E. (2009). Learning Multiple Layers of Features from Tiny Images, Citeseer. Technical Report.
20. Liu, Z., Luo, P., Wang, X., and Tang, X. (2015, January 7–13). Deep Learning Face Attributes in the Wild. Proceedings of the International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.425 ↗
21. Tero, K., Timo, A., Samuli, L., and Jaakko, L. (May, January 30). Progressive Growing of GANs for Improved Quality, Stability, and Variation. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada.
22. Dumoulin, V., Belghazi, I., Poole, B., Mastropietro, O., Lamb, A., Arjovsky, M., and Courville, A. (2016). Adversarially Learned Inference. arXiv.
23. Dieng, A.B., Ruiz, F.J., Blei, D.M., and Titsias, M.K. (2019). Prescribed Generative Adversarial Networks. arXiv.
24. Xiao, C., Zhong, P., and Zheng, C. (2018). BourGAN: Generative Networks with Metric Embeddings. Advances in Neural Information Processing Systems (NIPS), MIT Press.
25. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.C. (2017). Improved Training of Wasserstein GANs. Advances in Neural Information Processing Systems (NIPS), MIT Press.
26. Salimans, T., Zhang, H., Radford, A., and Metaxas, D. (2018). Improving GANs Using Optimal Transport. arXiv.
27. Martin Arjovsky, S.C., and Bottou, L. (2017). Wasserstein GAN. arXiv.
28. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems (NIPS), MIT Press.
29. Tim, S., Ian, G., Wojciech, Z., Vicki, C., Alec, R., and Chen, X. (2016). Improved Techniques for Training GANs. Advances in Neural Information Processing Systems (NIPS), MIT Press.
30. The frechet distance between multvariate normal distributions10.1016/0047-259X(82)90077-X ↗
31. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. (2017). Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems(NIPS), MIT Press.
32. Zhang, H., Goodfellow, I., Metaxas, D., and Odena, A. (2019, January 9–15). Self-Attention Generative Adversarial Networks. Proceedings of the International Conference on Machine Learning (ICLR), Long Beach, CA, USA.
33. Wei, X., Gong, B., Liu, Z., Lu, W., and Wang, L. (May, January 30). Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect. Proceedings of the International Conference on Machine Learning (ICLR), Vancouver, BC, Canada.
34. Liu, K., Tang, W., Zhou, F., and Qiu, G. (November, January 27). Spectral Regularization for Combating Mode Collapse in GANs. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.
35. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
36. Kumar, A., Sattigeri, P., and Fletcher, P.T. (2017). Improved Semi-supervised Learning with GANs using Manifold Invariances. arXiv.