Generative Enhancement of 3D Image Classifiers
Source abstract
Abstract
In this paper, we propose a methodology for generative enhancement of existing 3D image classifiers. This methodology is based on combining the advantages of both non-generative classifiers and generative modeling. Its purpose is to streamline the synthesis of novel deep neural networks by embedding existing compatible classifiers into a generative network architecture. A demonstration of this process and evaluation of its effectiveness is performed using a 3D convolutional classifier and its generative equivalent—a 3D conditional generative adversarial network classifier. The results of the experiments show that the generative classifier delivers higher performance, gaining a relative classification accuracy improvement of 7.43%. An increase of accuracy is also observed when comparing it to a plain convolutional classifier that was trained on a dataset augmented with samples created by the trained generator. This suggests a desirable knowledge sharing mechanism exists within the hybrid discriminator-classifier network.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Advanced Vision and Imaging · Computer Graphics and Visualization Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Cybernetics and Artificial Intelligence, FEI TU of Košice, 04200 Košice, Slovakia
Funding
Ministerstvo školstva, vedy, výskumu a športu Slovenskej republiky
References (49)
Show source-provided references
1. Hartmann, K.G., Schirrmeister, R.T., and Ball, T. (2018). EEG-GAN: Generative Adversarial Networks for Electroencephalograhic (EEG) Brain Signals. arXiv.
2. Chen, L., Dai, S., Tao, C., Zhang, H., Gan, Z., Shen, D., Zhang, Y., Wang, G., Zhang, R., and Carin, L. (2018, January 3–8). Adversarial Text Generation via Feature-Mover’s Distance. Proceedings of the Advances in Neural Information Processing Systems, Montréal, QC, Canada.
3. Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., and Metaxas, D.N. (2017, January 22–29). Stackgan: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.629 ↗
4. Pascual, S., Bonafonte, A., and Serra, J. (2017). SEGAN: Speech Enhancement Generative Adversarial Network. arXiv.10.21437/Interspeech.2017-1428 ↗
5. Vondrick, C., Pirsiavash, H., and Torralba, A. (2016, January 5–10). Generating Videos with Scene Dynamics. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain.
6. Wang, Y., Tao, X., Shen, X., and Jia, J. (2019, January 16–20). Wide-Context Semantic Image Extrapolation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00149 ↗
7. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21–26). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
8. Image Inpainting Based on Generative Adversarial Networks10.1109/ACCESS.2020.2970169 ↗
9. Shape2Vec: Semantic-Based Descriptors for 3D Shapes, Sketches and Images10.1145/2980179.2980253 ↗
10. Wu, J., Zhang, C., Xue, T., Freeman, B., and Tenenbaum, J. (2016, January 5–10). Learning a Probabilistic Latent Space of Object Shapes via 3d Generative-Adversarial Modeling. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain.
11. Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., and Weinberger, K.Q. (2014). Generative Adversarial Nets. Advances in Neural Information Processing Systems 27, Curran Associates, Inc.
12. Odena, A. (2016, January 19–24). Semi-Supervised Learning with Generative Adversarial Networks. Proceedings of the Workshop on Data-Efficient Machine Learning (ICML 2016), New York City, NY, USA.
13. Brock, A., Lim, T., Ritchie, J.M., and Weston, N.J. (2016, January 5–10). Generative and Discriminative Voxel Modeling with Convolutional Neural Networks. Proceedings of the Neural Inofrmation Processing Conference, Barcelona, Spain.
14. Achlioptas, P., Diamanti, O., Mitliagkas, I., and Guibas, L. (2018, January 10–15). Learning Representations and Generative Models for 3D Point Clouds. Proceedings of the International Conference on Machine Learning, PMLR, Stockholm, Sweden.
15. Khan, S.H., Guo, Y., Hayat, M., and Barnes, N. (2019, January 15–20). Unsupervised Primitive Discovery for Improved 3D Generative Modeling. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00997 ↗
16. Kanezaki, A., Matsushita, Y., and Nishida, Y. (2018, January 18–22). RotationNet: Joint Object Categorization and Pose Estimation Using Multiviews from Unsupervised Viewpoints. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00526 ↗
17. Han, Z., Wang, X., Vong, C.M., Liu, Y.S., Zwicker, M., and Chen, C.P. (2019, January 10–16). 3DViewGraph: Learning Global Features for 3D Shapes from a Graph of Unordered Views with Attention. Proceedings of the 2019 International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2019/107 ↗
18. Han, Z., Liu, X., Liu, Y.S., and Zwicker, M. (2019, January 10–16). Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2019/108 ↗
19. Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., and Xiao, J. (2015, January 7–12). 3d Shapenets: A Deep Representation for Volumetric Shapes. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.
20. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. (2016). Improved Techniques for Training GANs. Proceedings of the 30th International Conference on Neural Information Processing Systems, Curran Associates Inc.. NIPS’16.
21. Chintala, S. (2020, October 14). How to Train a GAN? Tips and Tricks to Make GANs Work. Available online: https://github.com/soumith/ganhacks.
22. Antoniou, A., Storkey, A., and Edwards, H. (2018). Data Augmentation Generative Adversarial Networks. arXiv.
23. The Effectiveness of Data Augmentation in Image Classification Using Deep Learning
24. Tanaka, F.H.K.d.S., and Aranha, C. (2019). Data Augmentation Using GANs. arXiv.
25. Generative Adversarial Network for Medical Images (MI-GAN)10.1007/s10916-018-1072-9 ↗
26. Židek, K., Lazorík, P., Pitel’, J., and Hošovskỳ, A. (2019). An Automated Training of Deep Learning Networks by 3D Virtual Models for Object Recognition. Symmetry, 11.10.3390/sym11040496 ↗
27. Conditional Image Synthesis With Auxiliary Classifier GANs
28. Mariani, G., Scheidegger, F., Istrate, R., Bekas, C., and Malossi, C. (2018). BAGAN: Data Augmentation with Balancing GAN. arXiv.
29. Sutskever, I., Jozefowicz, R., Gregor, K., Rezende, D., Lillicrap, T., and Vinyals, O. (2015). Towards Principled Unsupervised Learning. arXiv.
30. Springenberg, J.T. (2017, January 24–26). Unsupervised and Semi-Supervised Learning with Categorical Generative Adversarial Networks. Proceedings of the 5th International Conference on Learning Representations, Toulon, France.
31. Shen, P., Lu, X., Li, S., and Kawai, H. (2017, January 20–24). Conditional Generative Adversarial Nets Classifier for Spoken Language Identification. Proceedings of the INTERSPEECH 2017, Stockholm, Sweden.10.21437/Interspeech.2017-553 ↗
32. Fan, H., Su, H., and Guibas, L.J. (2017, January 21–26). A Point Set Generation Network for 3d Object Reconstruction from a Single Image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.264 ↗
33. Ren, M., Niu, L., and Fang, Y. (2017). 3D-A-Nets: 3D Deep Dense Descriptor for Volumetric Shapes with Adversarial Networks. arXiv.
34. Wang, W., Huang, Q., You, S., Yang, C., and Neumann, U. (2017, January 22–29). Shape Inpainting Using 3d Generative Adversarial Network and Recurrent Convolutional Networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.252 ↗
35. Yi, L., Zhao, W., Wang, H., Sung, M., and Guibas, L.J. (2019, January 16–20). Gspn: Generative Shape Proposal Network for 3d Instance Segmentation in Point Cloud. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00407 ↗
36. Wang, W., Yu, R., Huang, Q., and Neumann, U. (2018, January 18–22). Sgpn: Similarity Group Proposal Network for 3d Point Cloud Instance Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00272 ↗
37. Evaluation of Depth Modality in Convolutional Neural Network Classification of RGB-D Images10.15546/aeei-2018-0029 ↗
38. Jiang, J., Bao, D., Chen, Z., Zhao, X., and Gao, Y. (February, January 27). MLVCNN: Multi-Loop-View Convolutional Neural Network for 3D Shape Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
39. Bazrafkan, S., and Corcoran, P. (2018). Versatile Auxiliary Classifier with Generative Adversarial Network (Vac+ Gan), Multi Class Scenarios. arXiv.
40. Varga, M., and Jadlovský, J. (2018). Contribution to Generative Modelling and Classification of 3D Images. SCYR 2018: Proceedings from Conference, TUKE.
41. Min, P. (2020, October 14). Binvox. Available online: http://www.patrickmin.com/binvox.
42. Simplification and Repair of Polygonal Models Using Volumetric Techniques10.1109/TVCG.2003.1196006 ↗
43. Varga, M., and Jadlovský, J. (2019). Contribution to 3D Voxel Generative Adversarial Networks. SCYR 2019: Proceedings from Conference, TUKE.
44. Campagnola, L. (2020, October 14). PyQtGraph. Available online: http://www.pyqtgraph.org.
45. Mirza, M., and Osindero, S. (2014). Conditional Generative Adversarial Nets. arXiv.
46. Wasserstein GAN
47. Kingma, D.P., and Ba, J. (2015, January 7–9). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference for Learning Representations, San Diego, CA, USA.
48. Han, Z., Shang, M., Liu, Y.S., and Zwicker, M. (February, January 27). View Inter-Prediction GAN: Unsupervised Representation Learning for 3D Shapes by Learning Global Shape Memories to Support Local View Predictions. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
49. Jadlovský, J., Jadlovská, A., Jadlovská, S., Čerkala, J., Kopčík, M., Čabala, J., Oravec, M., Varga, M., and Vošček, D. (2016, January 21–23). Research Activities of the Center of Modern Control Techniques and Industrial Informatics. Proceedings of the 2016 IEEE 14th International Symposium on Applied Machine Intelligence and Informatics (SAMI), Herl’any, Slovakia.10.1109/SAMI.2016.7423021 ↗