Image-to-Image Translation Using Identical-Pair Adversarial Networks
Source abstract
Abstract
We propose Identical-pair Adversarial Networks (iPANs) to solve image-to-image translation problems, such as aerial-to-map, edge-to-photo, de-raining, and night-to-daytime. Our iPANs rely mainly on the effectiveness of adversarial loss function and its network architectures. Our iPANs consist of two main networks, an image transformation network T and a discriminative network D. We use U-NET for the transformation network T and a perceptual similarity network, which has two streams of VGG16 that share the same weights for network D. Our proposed adversarial losses play a minimax game against each other based on a real identical-pair and a fake identical-pair distinguished by the discriminative network D; e.g. a discriminative network D considers two inputs as a real pair only when they are identical, otherwise a fake pair. Meanwhile, the transformation network T tries to persuade the discriminator network D that the fake pair is a real pair. We experimented on several problems of image-to-image translation and achieved results that are comparable to those of some existing approaches, such as pix2pix, and PAN.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Processing Techniques · Generative Adversarial Networks and Image Synthesis · Digital Media Forensic Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Division of Computer Science and Engineering, Chonbuk National University, Jeonju 54896, Korea
Center for Advanced Image and Information Technology, Chonbuk National University, Jeonju 54896, Korea
Funding
National Research Foundation of Korea
References (40)
Show source-provided references
1. Bertalmio, M., Sapiro, G., Caselles, V., and Ballester, C. (2002, January 23–28). Image Inpainting. Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, USA.
2. Luan, Q., Wen, F., Cohen-Or, D., Liang, L., Xu, Y.-Q., and Shum, H.-Y. (2007, January 25–27). Natural image colorization. Proceedings of the 18th Eurographics Conference on Rendering Techniques, Grenoble, France.
3. Applications. Super-resolution: A comprehensive survey10.1007/s00138-014-0623-4 ↗
4. A survey: Image segmentation techniques10.7763/IJFCC.2014.V3.274 ↗
5. Image denoising via sparse and redundant representations over learned dictionaries10.1109/TIP.2006.881969 ↗
6. Clearing the skies: A deep network architecture for single-image rain removal10.1109/TIP.2017.2691802 ↗
7. Zhang, R., Isola, P., and Efros, A.A. (2016, January 11–14). Colorful image colorization. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46487-9_40 ↗
8. Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., and Efros, A.A. (July, January 26). Context encoders: Feature learning by inpainting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
9. Image super-resolution using deep convolutional networks10.1109/TPAMI.2015.2439281 ↗
10. Miyato, T., and Koyama, M. (2018). cGANs with projection discriminator. arXiv.
11. Perceptual adversarial networks for image-to-image transformation10.1109/TIP.2018.2836316 ↗
12. Liu, M.-Y., Breuel, T., and Kautz, J. (2017, January 4–9). Unsupervised image-to-image translation networks. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
13. Yi, Z., Zhang, H., Tan, P., and Gong, M. (2017, January 22–29). Dualgan: Unsupervised dual learning for image-to-image translation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.310 ↗
14. Lee, H.-Y., Tseng, H.-Y., Huang, J.-B., Singh, M., and Yang, M.-H. (2018, January 8–14). Diverse image-to-image translation via disentangled representations. Proceedings of the European Conference on Computer Vision (ECCV), Technical University of Munich, Munich, Germany.10.1007/978-3-030-01246-5_3 ↗
15. Murez, Z., Kolouri, S., Kriegman, D., Ramamoorthi, R., and Kim, K. (2018, January 18–22). Image to image translation for domain adaptation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00473 ↗
16. Zhu, J.-Y., Park, T., Isola, P., and Efros, A.A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv.10.1109/ICCV.2017.244 ↗
17. Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolutional networks for biomedical image segmentation. Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.
18. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–22). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00068 ↗
19. Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A.A. (2019, January 21–26). Image-to-image translation with conditional adversarial networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.632 ↗
20. Learning representations by back-propagating errors10.1038/323533a0 ↗
21. Noh, H., Hong, S., and Han, B. (2015, January 7–13). Learning deconvolution network for semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.178 ↗
22. Learning hierarchical features for scene labeling10.1109/TPAMI.2012.231 ↗
23. Eigen, D., and Fergus, R. (2015, January 7–13). Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.304 ↗
24. Context-aware patch-based image inpainting using Markov random field modeling10.1109/TIP.2014.2372479 ↗
25. Colorization Using Neural Network Ensemble10.1109/TIP.2017.2740620 ↗
26. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8–13). Generative adversarial nets. Proceedings of Advances in Neural Information Processing Systems, Montreal, QC, Canada.
27. Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., and Abbeel, P. (2016, January 5–10). Infogan: Interpretable representation learning by information maximizing generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems, Barcelona International Convention Centre, Barcelona, Spain.
28. Zhao, J., Mathieu, M., and LeCun, Y. (2016). Energy-based generative adversarial network. arXiv.
29. Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6–11). Wasserstein generative adversarial networks. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
30. Karras, T., Aila, T., Laine, S., and Lehtinen, J. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv.
31. Miyato, T., Kataoka, T., Koyama, M., and Yoshida, Y. (2018). Spectral normalization for generative adversarial networks. arXiv.
32. Zhang, H., Sindagi, V., and Patel, V.M. (2017). Image de-raining using a conditional generative adversarial network. arXiv.
33. Zhu, J.-Y., Krähenbühl, P., Shechtman, E., and Efros, A.A. (2016, January 8–16). Generative visual manipulation on the natural image manifold. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherland.10.1007/978-3-319-46454-1_36 ↗
34. Brock, A., Lim, T., Ritchie, J.M., and Weston, N. (2016). Neural photo editing with introspective adversarial networks. arXiv.
35. Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A.A. (2017, January 21–26). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.632 ↗
36. Lotter, W., Kreiman, G., and Cox, D. (2015). Unsupervised learning of visual structure using predictive generative networks. arXiv.
37. Zhu, J.-Y., Zhang, R., Pathak, D., Darrell, T., Efros, A.A., Wang, O., and Shechtman, E. (2017, January 4–9). Toward multimodal image-to-image translation. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
38. Image quality assessment: from error visibility to structural similarity10.1109/TIP.2003.819861 ↗
39. A universal image quality index10.1109/97.995823 ↗
40. Image information and visual quality10.1109/TIP.2005.859378 ↗