Survey on Implementations of Generative Adversarial Networks for Semi-Supervised Learning
Source abstract
Abstract
Given recent advances in deep learning, semi-supervised techniques have seen a rise in interest. Generative adversarial networks (GANs) represent one recent approach to semi-supervised learning (SSL). This paper presents a survey method using GANs for SSL. Previous work in applying GANs to SSL are classified into pseudo-labeling/classification, encoder-based, TripleGAN-based, two GAN, manifold regularization, and stacked discriminator approaches. A quantitative and qualitative analysis of the various approaches is presented. The R3-CGAN architecture is identified as the GAN architecture with state-of-the-art results. Given the recent success of non-GAN-based approaches for SSL, future research opportunities involving the adaptation of elements of SSL into GAN-based implementations are also identified.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Human Pose and Action Recognition · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer Science and Engineering Department, American University of Sharjah, Sharjah P.O. Box 26666, United Arab Emirates
References (58)
Show source-provided references
1. A Survey on Semi-Supervised Learning10.1007/s10994-019-05855-6 ↗
2. Unsupervised Clustering for Deep Learning: A Tutorial Survey10.12700/APH.15.8.2018.8.2 ↗
3. Ouali, Y., Hudelot, C., and Tami, M. (2020). An Overview of Deep Semi-Supervised Learning. arXiv.
4. Yang, X., Song, Z., King, I., and Xu, Z. (2021). A Survey on Deep Semi-Supervised Learning. arXiv.
5. Cowell, R.G., and Ghahramani, Z. (2005, January 6–8). Semi-Supervised Classification by Low Density Separation. Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, AISTATS 2005, Bridgetown, Barbados.
6. Krizhevsky, A. (2009). Learning Multiple Layers of Features from Tiny Images, University of Toronto. Technical report.
7. Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A.Y. (2011, January 12–17). Reading Digits in Natural Images with Unsupervised Feature Learning. Proceedings of the NIPS Workshop on Deep Learning and Unsupervised Feature Learning, Granada, Spain.
8. Sohn, K., Berthelot, D., Li, C.-L., Zhang, Z., Carlini, N., Cubuk, E.D., Kurakin, A., Zhang, H., and Raffel, C. (2020). FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence. arXiv.
9. Oliver, A., Odena, A., Raffel, C., Cubuk, E.D., and Goodfellow, I.J. (2018, January 3–8). Realistic Evaluation of Deep Semi-Supervised Learning Algorithms. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montréal, QC, Canada.
10. Zhang, H., Zhang, Z., Odena, A., and Lee, H. (2020, January 15–17). Consistency Regularization for Generative Adversarial Networks. Proceedings of the International Conference on Learning Representations, Shenzhen, China.
11. Sajjadi, M., Javanmardi, M., and Tasdizen, T. (2016, January 5–10). Regularization with Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain.
12. Laine, S., and Aila, T. (2017). Temporal Ensembling for Semi-Supervised Learning. arXiv.
13. Tarvainen, A., and Valpola, H. (2017, January 4–9). Mean Teachers Are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
14. Luo, Y., Zhu, J., Li, M., Ren, Y., and Zhang, B. (2018, January 18–23). Smooth Neighbors on Teacher Graphs for Semi-Supervised Learning. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00927 ↗
15. Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning10.1109/TPAMI.2018.2858821 ↗
16. Park, S., Park, J.-K., Shin, S.-J., and Moon, I.-C. (2017). Adversarial Dropout for Supervised and Semi-Supervised Learning. arXiv.10.1609/aaai.v32i1.11634 ↗
17. Verma, V., Lamb, A., Kannala, J., Bengio, Y., and Lopez-Paz, D. (2019, January 10–16). Interpolation Consistency Training for Semi-Supervised Learning. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2019/504 ↗
18. Berthelot, D., Carlini, N., Cubuk, E.D., Kurakin, A., Sohn, K., Zhang, H., and Raffel, C. (2020). ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring. arXiv.
19. Wei, X., Gong, B., Liu, Z., Lu, W., and Wang, L. (2018). Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect. arXiv.
20. Chen, Z., Ramachandra, B., and Vatsavai, R.R. (2020). Consistency Regularization with Generative Adversarial Networks for Semi-Supervised Learning. arXiv.
21. Zhao, Z., Singh, S., Lee, H., Zhang, Z., Odena, A., and Zhang, H. (2020). Improved Consistency Regularization for GANs. arXiv.10.1609/aaai.v35i12.17317 ↗
22. Lee, D.-H. (2013, January 21). Pseudo-Label: The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks. Proceedings of the Workshop on Challenges in Representation Learning, Atlanta, GA, USA.
23. Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (2018, January 8–14). Transductive Semi-Supervised Deep Learning Using Min-Max Features. Proceedings of the Computer Vision—ECCV 2018, Munich, Germany.10.1007/978-3-030-01252-6 ↗
24. Iscen, A., Tolias, G., Avrithis, Y., and Chum, O. (2019, January 15–20). Label Propagation for Deep Semi-Supervised Learning. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00521 ↗
25. Arazo, E., Ortego, D., Albert, P., O’Connor, N.E., and McGuinness, K. (2020). Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning. arXiv.10.1109/IJCNN48605.2020.9207304 ↗
26. Li, C., Xu, K., Zhu, J., and Zhang, B. (2017). Triple Generative Adversarial Nets. arXiv.
27. Wallach, H., Larochelle, H., Beygelzimer, A., Alché-Buc, F., Fox, E., and Garnett, R. (2019, January 8–14). MarginGAN: Adversarial Training in Semi-Supervised Learning. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
28. Liu, Y., Deng, G., Zeng, X., Wu, S., Yu, Z., and Wong, H.-S. (2020, January 13–19). Regularizing Discriminative Capability of CGANs for Semi-Supervised Generative Learning. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00576 ↗
29. Dai, Z., Yang, Z., Yang, F., Cohen, W.W., and Salakhutdinov, R. (2017). Good Semi-Supervised Learning That Requires a Bad GAN. arXiv.
30. Springenberg, J.T. (2016). Unsupervised and Semi-Supervised Learning with Categorical Generative Adversarial Networks. arXiv.
31. Odena, A. (2016). Semi-Supervised Learning with Generative Adversarial Networks. arXiv.
32. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. (2016). Improved Techniques for Training GANs. arXiv.
33. Sun, J., Bhattarai, B., and Kim, T.-K. (2020). MatchGAN: A Self-Supervised Semi-Supervised Conditional Generative Adversarial Network. arXiv.
34. Wu, S., Deng, G., Li, J., Li, R., Yu, Z., and Wong, H.-S. (2019, January 15–20). Enhancing TripleGAN for Semi-Supervised Conditional Instance Synthesis and Classification. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01033 ↗
35. Gan, Z., Chen, L., Wang, W., Pu, Y., Zhang, Y., Liu, H., Li, C., and Carin, L. (2017). Triangle Generative Adversarial Networks. arXiv.
36. Deng, Z., Zhang, H., Liang, X., Yang, L., Xu, S., Zhu, J., and Xing, E.P. (2017). Structured Generative Adversarial Networks. arXiv.
37. Haque, A. (2021). EC-GAN: Low-Sample Classification Using Semi-Supervised Algorithms and GANs. arXiv.10.1609/aaai.v35i18.17895 ↗
38. Donahue, J., Krähenbühl, P., and Darrell, T. (2017). Adversarial Feature Learning. arXiv.
39. Dumoulin, V., Belghazi, I., Poole, B., Mastropietro, O., Lamb, A., Arjovsky, M., and Courville, A. (2017). Adversarially Learned Inference. arXiv.
40. Kumar, A., Sattigeri, P., and Fletcher, P.T. (2017). Semi-Supervised Learning with GANs: Manifold Invariance with Improved Inference. arXiv.
41. Lecouat, B., Foo, C.-S., Zenati, H., and Chandrasekhar, V.R. (2018). Semi-Supervised Learning with GANs: Revisiting Manifold Regularization. arXiv.
42. Lecouat, B., Foo, C.-S., Zenati, H., and Chandrasekhar, V. (2018). Manifold Regularization with GANs for Semi-Supervised Learning. arXiv.
43. Attention-Based Generative Adversarial Network for Semi-Supervised Image Classification10.1007/s11063-019-10158-x ↗
44. Tang, X., Yu, X., Xu, J., Chen, Y., and Wang, R. (2020, January 22–24). Semi-Supervised Generative Adversarial Networks Based on Scalable Support Vector Machines and Manifold Regularization. Proceedings of the 2020 Chinese Control And Decision Conference (CCDC), Hefei, China.10.1109/CCDC49329.2020.9164865 ↗
45. Motamed, S., and Khalvati, F. (2021). Multi-Class Generative Adversarial Nets for Semi-Supervised Image Classification. arXiv.
46. Motamed, S., and Khalvati, F. (2021). Vanishing Twin GAN: How Training a Weak Generative Adversarial Network Can Improve Semi-Supervised Image Classification. arXiv.
47. Motamed, S., Rogalla, P., and Khalvati, F. (2021). Data Augmentation Using Generative Adversarial Networks (GANs) for GAN-Based Detection of Pneumonia and COVID-19 in Chest X-Ray Images. arXiv.10.21203/rs.3.rs-146161/v1 ↗
48. Sricharan, K., Bala, R., Shreve, M., Ding, H., Saketh, K., and Sun, J. (2017). Semi-Supervised Conditional GANs. arXiv.
49. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative Adversarial Networks. arXiv.
50. Arjovsky, M., Chintala, S., and Bottou, L. (2017). Wasserstein GAN. arXiv.
51. Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., and Yoo, Y. (2019). CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features. arXiv.10.1109/ICCV.2019.00612 ↗
52. Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., and Langs, G. (2017). Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery. arXiv.10.1007/978-3-319-59050-9_12 ↗
53. Mirza, M., and Osindero, S. (2014). Conditional Generative Adversarial Nets. arXiv.
54. Li, W., Wang, Z., Li, J., Polson, J., Speier, W., and Arnold, C. (2019). Semi-Supervised Learning Based on Generative Adversarial Network: A Comparison between Good GAN and Bad GAN Approach. arXiv.
55. Cubuk, E.D., Zoph, B., Shlens, J., and Le, Q.V. (2019). RandAugment: Practical Automated Data Augmentation with a Reduced Search Space. arXiv.10.1109/CVPRW50498.2020.00359 ↗
56. Cubuk, E.D., Zoph, B., Mané, D., Vasudevan, V., and Le, Q.V. (2019, January 15–20). AutoAugment: Learning Augmentation Strategies From Data. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00020 ↗
57. Xie, Q., Dai, Z., Hovy, E., Luong, M.-T., and Le, Q.V. (2020). Unsupervised Data Augmentation for Consistency Training. arXiv.
58. Survey on Deep Learning with Class Imbalance10.1186/s40537-019-0192-5 ↗