A Small-Sample Text Classification Model Based on Pseudo-Label Fusion Clustering Algorithm
Source abstract
Abstract
The problem of text classification has been a mainstream research branch in natural language processing, and how to improve the effect of classification under the scarcity of labeled samples is one of the hot issues in this direction. The current models supporting small-sample classification can learn knowledge and train models with a small number of labels, but the classification results are not satisfactory enough. In order to improve the classification accuracy, we propose a Small-sample Text Classification model based on the Pseudo-label fusion Clustering algorithm (STCPC). The algorithm includes two cores: (1) Mining the potential features of unlabeled data by using the training strategy of clustering assuming pseudo-labeling and then reducing the noise of the pseudo-labeled dataset by consistent training with its enhanced samples to improve the quality of the pseudo-labeled dataset. (2) The labeled data is augmented, and then the Easy Plug-in Data Augmentation (EPiDA) framework is used to balance the diversity and quality of the augmented samples to improve the richness of the labeled data reasonably. The results of comparison tests with other classical algorithms show that the STCPC model can effectively improve classification accuracy.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Text and Document Classification Technologies · Web Data Mining and Analysis · Image Retrieval and Classification Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer, Electronic and Information, Guangxi University, Nanning 530004, China
Funding
National Natural Science Foundation of China
References (40)
Show source-provided references
1. Nasukawa, T., and Yi, J. (2003, January 23–25). Sentiment Analysis: Capturing Favorability Using Natural Language Processing. Proceedings of the 2nd International Conference on Knowledge Capture, Sanibel Island, FL, USA.10.1145/945645.945658 ↗
2. Ma, L., and Zhang, Y. (November, January 29). Using Word2Vec to Process Big Text Data. Proceedings of the 2015 IEEE International Conference on Big Data (Big Data), Santa Clara, CA, USA.10.1109/BigData.2015.7364114 ↗
3. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. arXiv.
4. Clark, K., Luong, M.-T., Le, Q.V., and Manning, C.D. (2020). ELECTRA: Pre-Training Text Encoders as Discriminators Rather than Generators. arXiv.
5. Yang, M. (2021, January 14–16). A Survey on Few-Shot Learning in Natural Language Processing. Proceedings of the 2021 International Conference on Artificial Intelligence and Electromechanical Automation (AIEA), Guangzhou, China.10.1109/AIEA53260.2021.00069 ↗
6. Howard, J., and Ruder, S. (2018). Universal Language Model Fine-Tuning for Text Classification. arXiv.10.18653/v1/P18-1031 ↗
7. Nguyen-Nhat, D.-K., and Duong, H.-T. (2019). Computational Data and Social Networks, Springer International Publishing.
8. A Review: Preprocessing Techniques and Data Augmentation for Sentiment Analysis10.1186/s40649-020-00080-x ↗
9. Do Not Have Enough Data? Deep Learning to the Rescue!
10. Zhao, M., Zhang, L., Xu, Y., Ding, J., Guan, J., and Zhou, S. (2022). EPiDA: An Easy Plug-in Data Augmentation Framework for High Performance Text Classification. arXiv.10.18653/v1/2022.naacl-main.349 ↗
11. Hidetaka, I., and Wang, Y. (2019, January 9–12). A Semi-Supervised Approach for Identification of the Sections in Charge of RFQ Documents. Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA.10.1109/BigData47090.2019.9006425 ↗
12. Jedrzejowicz, J., and Zakrzewska, M. (2020). Intelligent Decision Technologies 2019, Springer Singapore.
13. Zoph, B., Ghiasi, G., Lin, T.-Y., Cui, Y., Liu, H., Cubuk, E.D., and Le, Q.V. (2020). Rethinking Pre-Training and Self-Training. arXiv.
14. Villatoro-Tello, E., Anguiano, E., Montes-y-Gómez, M., Villaseñor-Pineda, L., and Ramírez-de-la-Rosa, G. (2016). Lecture Notes in Computer Science, Springer International Publishing.
15. Zhang, Z., Luo, J., and Huang, G. (2019, January 12–14). A Semi-Supervised Short Text Classification Method Based on Weighted Word Vector Representation. Proceedings of the 2019 IEEE 9th International Conference on Electronics Information and Emergency Communication (ICEIEC), Beijing, China.10.1109/ICEIEC.2019.8784604 ↗
16. Semisupervised Deep Convolutional Neural Networks Using Pseudo Labels for PolSAR Image Classification
17. Generalizing from a Few Examples: A Survey on Few-Shot Learning
18. Learning from Examples in the Small Sample Case: Face Expression Recognition10.1109/TSMCB.2005.846658 ↗
19. Discriminant Common Vectors versus Neighbourhood Components Analysis and Laplacianfaces: A Comparative Study in Small Sample Size Problem10.1016/j.imavis.2005.11.007 ↗
20. Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories10.1016/j.cviu.2005.09.012 ↗
21. Maas, A., and Kemp, C. (August, January 29). One-Shot Learning with Bayesian Networks. Proceedings of the Annual Meeting of the Cognitive Science Society, Amsterdam, The Netherlands. No. 31.
22. Foundations of Statistical Natural Language Processing (Review)10.1353/lan.2002.0150 ↗
23. Nakamura, A., and Harada, T. (2019). Revisiting Fine-Tuning for Few-Shot Learning. arXiv.
24. Wei, J., and Zou, K. (2019). EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks. arXiv.10.18653/v1/D19-1670 ↗
25. Yu, F., Wang, H., Zheng, D., and Fei, G. (2010, January 23–24). Research on Transfer Learning Approach for Text Categorization. Proceedings of the 2010 International Conference on Artificial Intelligence and Computational Intelligence, Sanya, China.10.1109/AICI.2010.94 ↗
26. Snell, J., Swersky, K., and Zemel, R.S. (2017). Prototypical Networks for Few-Shot Learning. arXiv.
27. Li, H., Eigen, D., Dodge, S., Zeiler, M., and Wang, X. (2019, January 15–20). Finding Task-Relevant Features for Few-Shot Learning by Category Traversal. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00009 ↗
28. Goel, A., Jiao, Y., and Massiah, J. (2022). PARS: Pseudo-Label Aware Robust Sample Selection for Learning with Noisy Labels. arXiv.
29. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks
30. Rizve, M.N., Duarte, K., Rawat, Y.S., and Shah, M. (2021). In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-Label Selection Framework for Semi-Supervised Learning. arXiv.
31. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. arXiv.
32. Koroteev, M.V. (2021). BERT: A Review of Applications in Natural Language Processing and Understanding. arXiv.
33. Yang, D., and Zhang, A.N. (2018, January 10–13). Performing Literature Review Using Text Mining, Part III: Summarizing Articles Using TextRank. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.10.1109/BigData.2018.8622408 ↗
34. Li, M., and Yuan, B. (September, January 31). A Novel Statistical Linear Discriminant Analysis for Image Matrix: Two-Dimensional Fisherfaces. Proceedings of the 7th International Conference on Signal Processing, 2004. Proceedings. ICSP ’04. 2004, Beijing, China.
35. A Simple Probabilistic Explanation of Term Frequency-Inverse Document Frequency (Tf-Idf) Heuristic (and Variations Motivated by This Explanation)10.1080/03081079.2017.1291635 ↗
36. A Survey of Clustering Algorithms for Big Data: Taxonomy and Empirical Analysis10.1109/TETC.2014.2330519 ↗
37. K-Means Clustering Algorithms: A Comprehensive Review, Variants Analysis, and Advances in the Era of Big Data10.1016/j.ins.2022.11.139 ↗
38. Bock, H.-H. (2007). Selected Contributions in Data Analysis and Classification, Springer.
39. Tarvainen, A., and Valpola, H. (2017). Mean Teachers Are Better Role Models: Weight-Averaged Consistency Targets Improve Semi-Supervised Deep Learning Results. arXiv.
40. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv.