Synthetic Data Augmentation for Imbalanced Tabular Protein Subcellular Localization: A Comparative Study of SMOTE, CTGAN, TVAE, and TabDDPM Methods
Source abstract
Abstract
Class imbalance is a persistent challenge in supervised machine learning, particularly in biological datasets where minority classes represent functionally critical categories. Synthetic data generation has emerged as a principal strategy for mitigating this problem, yet systematic comparisons of classical and modern deep generative approaches remain limited. This study presents a comprehensive benchmark evaluation of four synthetic data generation methods—SMOTE, CTGAN, TVAE, and TabDDPM—across two well-established biological datasets from the UCI Machine Learning Repository: the E. coli protein localization dataset (307 samples, 6 features, 4 classes) and the yeast protein localization dataset (1299 samples, 8 features, 4 classes). Synthetic data quality was rigorously assessed using a multi-dimensional evaluation framework encompassing distributional fidelity (Fréchet Distance, Wasserstein Distance), machine learning utility (Train-on-Synthetic-Test-on-Real and Train-on-Real-Test-on-Real protocols using XGBoost version 3.2.0, Logistic Regression, Support Vector Machines, and Random Forest), and distinguishability (Classifier Two-Sample Test). The datasets are rather imbalanced. During the experiments, the dataset size increased to three times its original size while preserving the imbalanced class-sample ratio. To evaluate the quality of synthetic data, the max(AUC,1−AUC) score is proposed. This score is inversely proportional to classification performance, indicating that synthetic data are not easily distinguishable from real data. Per-class analysis reveals that minority classes remain the primary challenge across all generative methods. SMOTE and TabDDPM obtained the highest predictive utility F1-scores across both datasets. TVAE offers the strongest distributional fidelity among deep generative models, producing synthetic samples that are most difficult to distinguish from real data (lowest C2ST scores). CTGAN exhibits significant performance degradation on both small- and medium-scale datasets, with F1 utility ratios below 0.50.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Imbalanced Data Classification Techniques · Machine Learning in Bioinformatics · Digital Imaging for Blood Diseases
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Software Engineering, Malatya Turgut Özal University, Malatya 44900, Turkey
References (42)
Show source-provided references
1. Learning from Imbalanced Data10.1109/TKDE.2008.239 ↗
2. Blagus, R., and Lusa, L. (2013). SMOTE for High-Dimensional Class-Imbalanced Data. BMC Bioinform., 14.10.1186/1471-2105-14-106 ↗
3. SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15th Anniversary10.1613/jair.1.11192 ↗
4. SMOTE: Synthetic Minority Over-sampling Technique10.1613/jair.953 ↗
5. Xu, L., Skoularidou, M., Cuesta-Infante, A., and Veeramachaneni, K. (2019, January 8–14). Modeling Tabular Data Using Conditional GAN. Proceedings of the Advances in Neural Information Processing Systems 32 (NeurIPS 2019), Vancouver, BC, Canada.
6. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8–13). Generative Adversarial Nets. Proceedings of the Advances in Neural Information Processing Systems 27 (NeurIPS 2014), Montreal, QC, Canada.
7. Kotelnikov, A., Baranchuk, D., Rubachev, I., and Babenko, A. (2023, January 23–29). TabDDPM: Modeling Tabular Data with Diffusion Models. Proceedings of the 40th International Conference on Machine Learning (ICML 2023), Honolulu, HI, USA.
8. A Study of the Behavior of Several Methods for Balancing Machine Learning Training Data10.1145/1007730.1007735 ↗
9. Han, H., Wang, W.Y., and Mao, B.H. (2005, January 23–26). Borderline-SMOTE: A New Over-Sampling Method in Learning from Imbalanced Data Sets. Proceedings of the International Conference on Intelligent Computing (ICIC 2005), Hefei, China.10.1007/11538059_91 ↗
10. He, H., Bai, Y., García, E.A., and Li, S. (2008, January 1–8). ADASYN: Adaptive Synthetic Sampling Approach for Imbalanced Learning. Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN 2008), Hong Kong, China.
11. Cancer prediction in imbalanced datasets: RUS or SMOTEEN for class imbalance?
12. Data Synthesis Based on Generative Adversarial Networks10.14778/3231751.3231757 ↗
13. Zhao, Z., Kunar, A., Birke, R., and Chen, L.Y. (2021, January 17–19). CTAB-GAN: Effective Table Data Synthesizing. Proceedings of the Asian Conference on Machine Learning (ACML 2021), Virtual.
14. Conditional Wasserstein GAN-Based Oversampling of Tabular Data for Imbalanced Learning10.1016/j.eswa.2021.114582 ↗
15. Kingma, D.P., and Welling, M. (2014, January 14–16). Auto-Encoding Variational Bayes. Proceedings of the 2nd International Conference on Learning Representations (ICLR 2014), Banff, AB, Canada.
16. Ma, C., Tschiatschek, S., Turner, R., Hernández-Lobato, J.M., and Zhang, C. (2020, January 6–12). VAEM: A Deep Generative Model for Heterogeneous Mixed Type Data. Proceedings of the Advances in Neural Information Processing Systems 33 (NeurIPS 2020), Virtual.
17. Zhang, Y., Zaidi, N.A., Zhou, J., and Li, G. (2022, January 28–30). GANBLR++: Incorporating Capacity to Generate Numeric Attributes and Leveraging Unrestricted Bayesian Networks. Proceedings of the 2022 SIAM International Conference on Data Mining (SDM 2022), Alexandria, VA, USA.10.1137/1.9781611977172.34 ↗
18. Ho, J., Jain, A., and Abbeel, P. (2020, January 6–12). Denoising Diffusion Probabilistic Models. Proceedings of the Advances in Neural Information Processing Systems 33 (NeurIPS 2020), Virtual.
19. Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. (2015, January 6–11). Deep Unsupervised Learning Using Nonequilibrium Thermodynamics. Proceedings of the 32nd International Conference on Machine Learning (ICML 2015), Lille, France.
20. Kim, J., Lee, C., and Park, N. (2023, January 1–5). STaSy: Score-Based Tabular Data Synthesis. Proceedings of the 11th International Conference on Learning Representations (ICLR 2023), Kigali, Rwanda.
21. Lee, C., Kim, J., and Park, N. (2023, January 23–29). CoDi: Co-Evolving Contrastive Diffusion Models for Mixed-Type Tabular Synthesis. Proceedings of the 40th International Conference on Machine Learning (ICML 2023), Honolulu, HI, USA.
22. Deep Neural Networks and Tabular Data: A Survey10.1109/TNNLS.2022.3229161 ↗
23. Jolicoeur-Martineau, A., Piché-Taillefer, R., and Mitliagkas, I. (2023). Generating and Imputing Tabular Data via Diffusion and Flow-Based Gradient-Boosted Trees. arXiv.
24. Dankar, F.K., and Ibrahim, M. (2021). Fake It till You Make It: Guidelines for Effective Synthetic Data Generation. Appl. Sci., 11.10.3390/app11052158 ↗
25. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. (2017, January 4–9). GANs Trained by a Two-Time-Scale Update Rule Converge to a Local Nash Equilibrium. Proceedings of the Advances in Neural Information Processing Systems 30 (NeurIPS 2017), Long Beach, CA, USA.
26. Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6–11). Wasserstein Generative Adversarial Networks. Proceedings of the 34th International Conference on Machine Learning (ICML 2017), Sydney, NSW, Australia.
27. Esteban, C., Hyland, S.L., and Rätsch, G. (2017). Real-Valued (Medical) Time Series Generation with Recurrent Conditional GANs. arXiv.
28. Lopez-Paz, D., and Oquab, M. (2017, January 24–26). Revisiting Classifier Two-Sample Tests. Proceedings of the 5th International Conference on Learning Representations (ICLR 2017), Toulon, France.
29. Figueira, A., and Vaz, B. (2022). Survey on Synthetic Data Generation, Evaluation Methods and GANs. Mathematics, 10.10.3390/math10152733 ↗
30. Horton, P., and Nakai, K. (1996, January 12–15). A Probabilistic Classification System for Predicting the Cellular Localization Sites of Proteins. Proceedings of the Fourth International Conference on Intelligent Systems for Molecular Biology (ISMB 1996), St. Louis, MO, USA.
31. KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework
32. Imbalanced-Learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning
33. Mirza, M., and Osindero, S. (2014). Conditional Generative Adversarial Nets. arXiv, arXiv:1411.1784.
34. Patki, N., Wedge, R., and Veeramachaneni, K. (2016, January 17–19). The Synthetic Data Vault. Proceedings of the IEEE International Conference on Data Science and Advanced Analytics (DSAA 2016), Montreal, QC, Canada.10.1109/DSAA.2016.49 ↗
35. Kingma, D.P., and Ba, J. (2015, January 7–9). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), San Diego, CA, USA.
36. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
37. Ho, J., and Salimans, T. (2022). Classifier-Free Diffusion Guidance. arXiv.
38. Scikit-Learn: Machine Learning in Python
39. Chen, T., and Guestrin, C. (2016, January 13–17). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2016), San Francisco, CA, USA.10.1145/2939672.2939785 ↗
40. Hosmer, D.W., Lemeshow, S., and Sturdivant, R.X. (2013). Applied Logistic Regression, Wiley. [3rd ed.].10.1002/9781118548387 ↗
41. Support-vector networks10.1023/A:1022627411411 ↗
42. Random Forests10.1023/A:1010933404324 ↗