Iterative Semi-Supervised Learning Using Softmax Probability
Source abstract
Abstract
For the classification problem in practice, one of the challenging issues is to obtain enough labeled data for training. Moreover, even if such labeled data has been sufficiently accumulated, most datasets often exhibit long-tailed distribution with heavy class imbalance, which results in a biased model towards a majority class. To alleviate such class imbalance, semi-supervised learning methods using additional unlabeled data have been considered. However, as a matter of course, the accuracy is much lower than that from supervised learning. In this study, under the assumption that additional unlabeled data is available, we propose the iterative semi-supervised learning algorithms, which iteratively correct the labeling of the extra unlabeled data based on softmax probabilities. The results show that the proposed algorithms provide the accuracy as high as that from the supervised learning. To validate the proposed algorithms, we tested on the two scenarios: with the balanced unlabeled dataset and with the imbalanced unlabeled dataset. Under both scenarios, our proposed semi-supervised learning algorithms provided higher accuracy than previous state-of-the-arts. Code is available at .
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Imbalanced Data Classification Techniques · Machine Learning and Data Classification · Domain Adaptation and Few-Shot Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (39)
Show source-provided references
1. A survey of image classification methods and techniques
2. A survey of medical image classification techniques
3. Vocaturo · Handbook of Research on Disease Prediction Through Data Analytics and Machine Learning · 2021 · 2210.4018/978-1-7998-2742-9.ch003 ↗
4. Analysis of artificial intelligence based image classification techniques10.36548/jiip.2020.1.005 ↗
5. TBE-Net: A three-branch embedding network with part-aware ability and feature complementary learning for vehicle re-identification10.1109/TITS.2021.3130403 ↗
6. RSOD: Real-time small object detection algorithm in UAV-based traffic monitoring
7. Imagenet classification with deep convolutional neural networks
8. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014. [Online]. Available: https://arxiv.org/abs/1409.1556.
9. Going deeper with convolutions
10. Deep residual learning for image recognition
11. Rethinking the inception architecture for computer vision
12. Physician time spent using the electronic health record during outpatient encounters: A descriptive study10.7326/M18-3684 ↗
13. Preparing medical imaging data for machine learning10.1148/radiol.2020192224 ↗
14. Deep learning of feature representation with multiple instance learning for medical image analysis
15. A survey on semi-supervised learning10.1007/s10994-019-05855-6 ↗
16. Virtual adversarial training: A regularization method for supervised and semi-supervised learning10.1109/TPAMI.2018.2858821 ↗
17. Semi-supervised learning for network-based cardiac MR image segmentation
18. Learning from imbalanced data: Open challenges and future directions10.1007/s13748-016-0094-0 ↗
19. A systematic review on imbalanced data challenges in machine learning: Applications and solutions
20. Learning pattern classification tasks with imbalanced data sets
21. Deep learning for imbalanced multimedia data classification
22. Deep learning for imbalanced multimedia data classification
23. Special issue on learning from imbalanced data sets10.1145/1007730.1007733 ↗
24. Survey on deep learning with class imbalance10.1186/s40537-019-0192-5 ↗
25. SMOTE: Synthetic minority over-sampling technique10.1613/jair.953 ↗
26. ADASYN: Adaptive synthetic sampling approach for imbalanced learning
27. The foundations of cost-sensitive learning
28. Cost-sensitive learning and the class imbalance problem
29. Class-balanced loss based on effective number of samples
30. Y. Yang and Z. Xu, “Rethinking the value of labels for improving class-imbalanced learning,” 2020. [Online]. Available: https://arxiv.org/abs/2006.07529.
31. K. Cao, C. Wei, A. Gaidon, N. Arechiga and T. Ma, “Learning imbalanced datasets with label-distribution-aware margin loss,” 2019. [Online]. Available: https://arxiv.org/abs/1906.07413.
32. Reading digits in natural images with unsupervised feature learning
33. 80 Million tiny images: A large data set for nonparametric object and scene recognition10.1109/TPAMI.2008.128 ↗
34. B. Kang, S. Xie, M. Rohrbach, Z. Yan, A. Gordo et al., “Decoupling representation and classifier for long-tailed recognition,” 2019. [Online]. Available: https://arxiv.org/abs/1910.09217.
35. Large-scale long-tailed recognition in an open world
36. S. Laine and T. Aila, “Temporal ensembling for semi-supervised learning,” 2016. [Online]. Available: https://arxiv.org/abs/1610.02242.
37. A. Tarvainen and H. Valpola, “Weight-averaged consistency targets improve semi-supervised deep learning results,” CoRR, 2017. [Online]. Available: https://arxiv.org/abs/1703.01780.
38. V. Verma, K. Kawaguchi, A. Lamb, J. Kannala, Y. Bengio et al., “Interpolation consistency training for semi-supervised learning,” 2019. [Online]. Available: https://arxiv.org/abs/1903.03825.10.24963/ijcai.2019/504 ↗
39. K. Sohn, D. Berthelot, C.-L. Li, Z. Zhang, N. Carlini et al., “Fixmatch: Simplifying semi-supervised learning with consistency and confidence,” 2020. [Online]. Available: https://arxiv.org/abs/2001.07685.