A Feature Selection Method for Multi-Label Text Based on Feature Importance
Source abstract
Abstract
Multi-label text classification refers to a text divided into multiple categories simultaneously, which corresponds to a text associated with multiple topics in the real world. The feature space generated by text data has the characteristics of high dimensionality and sparsity. Feature selection is an efficient technology that removes useless and redundant features, reduces the dimension of the feature space, and avoids dimension disaster. A feature selection method for multi-label text based on feature importance is proposed in this paper. Firstly, multi-label texts are transformed into single-label texts using the label assignment method. Secondly, the importance of each feature is calculated using the method based on Category Contribution (CC). Finally, features with higher importance are selected to construct the feature space. In the proposed method, the feature importance is calculated from the perspective of the category, which ensures the selected features have strong category discrimination ability. Specifically, the contributions of the features to each category from two aspects of inter-category and intra-category are calculated, then the importance of the features is obtained with the combination of them. The proposed method is tested on six public data sets and the experimental results are good, which demonstrates the effectiveness of the proposed method.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Text and Document Classification Technologies · Web Data Mining and Analysis · Spam and Phishing Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Information and Electrical Engineering, China Agricultural University, Beijing 100083, China
Funding
the Monitoring and Statistical Projects of Agricultural and Rural Resources · the Integrated Application of Core Information Technology for Early Warning, Diagnosis and Prevention of Facility Vegetable Diseases · the National High Technology Research and Development Program of China (863 Program)
References (44)
Show source-provided references
1. Classification technique of Chinese agricultural text information based on SVM
2. Ren, F., and Deng, J. (2018). Background Knowledge Based Multi-Stream Neural Network for Text Classification. Appl. Sci., 8.10.3390/app8122472 ↗
3. Toward an enhanced Arabic text classification using cosine similarity and Latent Semantic Indexing
4. Labelset topic model for multi-label document classification10.1007/s10844-014-0352-1 ↗
5. Liu, J., Chang, W., Wu, Y., and Yang, Y. (2017, January 7–11). Deep Learning for Extreme Multi-label Text Classification. Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, Tokyo, Japan.10.1145/3077136.3080834 ↗
6. Liu, P., Qiu, X., and Huang, X. (August, January 30). Adversarial Multi-task Learning for Text Classification. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics 2017, Vancouver, BC, Canada.10.18653/v1/P17-1001 ↗
7. Guo, Y., Chung, F., and Li, G. (2016, January 15–18). An ensemble embedded feature selection method for multi-label clinical text classification. Proceedings of the 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Shenzhen, China.
8. Glinka, K., Wozniak, R., and Zakrzewska, D. (2017, January 21–23). Improving Multi-label Medical Text Classification by Feature Selection. Proceedings of the 2017 IEEE 26th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE), Poznan, Poland.10.1109/WETICE.2017.42 ↗
9. Feature selection for multi-label naive Bayes classification10.1016/j.ins.2009.06.010 ↗
10. Symptom selection for multi-label data of inquiry diagnosis in traditional Chinese medicine10.1007/s11432-011-4406-5 ↗
11. Yu, Y., and Wang, Y. (2014). Feature selection for multi-label learning using mutual information and GA. International Conference on Rough Sets and Knowledge Technology, Springer.10.1007/978-3-319-11740-9_42 ↗
12. Gharroudi, Q., Elghazel, H., and Aussem, A. (2014). A Comparison of Multi-Label Feature Selection Methods Using the Random Forest Paradigm. Advances in Artificial Intelligence, Springer.10.1007/978-3-319-06483-3_9 ↗
13. Memetic feature selection algorithm for multi-label classification10.1016/j.ins.2014.09.020 ↗
14. Gu, Q., Li, Z., and Han, J. (2011, January 24–28). Correlated multi-label feature selection. Proceedings of the ACM International Conference on Information and Knowledge Management, Glasgow, UK.10.1145/2063576.2063734 ↗
15. Embedded Feature Selection for Multi-label Classification of Music Emotions10.1080/18756891.2012.718113 ↗
16. Multi-label feature selection via feature manifold learning and sparsity regularization10.1007/s13042-017-0647-y ↗
17. Xu, H., and Xu, L. (2017, January 14–19). Multi-label feature selection algorithm based on label pairwise ranking comparison transformation. Proceedings of the International Joint Conference on Neural Networks, Anchorage, AK, USA.10.1109/IJCNN.2017.7965990 ↗
18. Feature selection for multi-label classification using multivariate mutual information10.1016/j.patrec.2012.10.005 ↗
19. Mutual information-based feature selection for multilabel classification10.1016/j.neucom.2013.06.035 ↗
20. Multi-label feature selection based on max-dependency and min-redundancy10.1016/j.neucom.2015.06.010 ↗
21. Feature selection for text classification: A review10.1007/s11042-018-6083-5 ↗
22. Largeron, C., Moulin, C., and Géry, M. (2011, January 21–24). Entropy based feature selection for text categorization. Proceedings of the 2011 ACM Symposium on Applied Computing, TaiChung, Taiwan.10.1145/1982185.1982389 ↗
23. A Feature Selection Approach Based on Interclass and Intraclass Relative Contributions of Terms10.1155/2016/1715780 ↗
24. Spyromitros, E., Tsoumakas, G., and Vlahavas, I. (2008). An Empirical Study of Lazy Multilabel Classification Algorithms, Springer.
25. ML-KNN: A lazy learning approach to multi-label learning10.1016/j.patcog.2006.12.019 ↗
26. Lastra, G., Luaces, O., Quevedo, J.R., and Bahamonde, A. (2011, January 29–31). Graphical Feature Selection for Multilabel Classification Tasks. Proceedings of the Advances in Intelligent Data Analysis X-international Symposium, Porto, Portugal.10.1007/978-3-642-24800-9_24 ↗
27. Granular multi-label feature selection based on mutual information10.1016/j.patcog.2017.02.025 ↗
28. Efficient Feature Selection via Analysis of Relevance and Redundancy
29. Chen, W., Yan, J., Zhang, B., Chen, Z., and Yang, Q. (2007, January 21–31). Document Transformation for Multi-label Feature Selection in Text Categorization. Proceedings of the IEEE International Conference on Data Mining, Omaha, NE, USA.10.1109/ICDM.2007.18 ↗
30. Multi-label classification of music by emotion10.1186/1687-4722-2011-426793 ↗
31. A Comparison of Multi-label Feature Selection Methods using the Problem Transformation Approach10.1016/j.entcs.2013.02.010 ↗
32. A systematic review of multi-label feature selection and a new method based on label construction10.1016/j.neucom.2015.07.118 ↗
33. Doquire, G., and Verleysen, M. (2011). Feature Selection for Multi-label Classification Problems, Springer.10.1155/2011/643816 ↗
34. Multi-label feature selection based on neighborhood mutual information10.1016/j.asoc.2015.10.009 ↗
35. A Comparative Study on Feature Selection in Text Categorization
36. Word association norms, mutual information, and lexicography
37. A mathematical theory of communication10.1002/j.1538-7305.1948.tb01338.x ↗
38. Van Rijsbergen, C. (1979). Information Retrieval, Butterworth-Heinemann.
39. Pestian, J.P., Brew, C., Matykiewicz, P., Hovermale, D.J., Johnson, N., Cohen, K.B., and Duch, W. (2007, January 29). A shared task involving multi-label classification of clinical free text. Proceedings of the Workshop on BioNLP 2007: Biological, Translational, and Clinical Language Processing, Prague, Czech Republic.10.3115/1572392.1572411 ↗
40. Ueda, N., and Saito, K. (2002). Parametric mixture models for multi-labeled text. International Conference on Neural Information Processing Systems, MIT Press.
41. BoosTexter: A boosting-based system for text categorization10.1023/A:1007649029923 ↗
42. Joint learning of multi-label classification and label correlations
43. Term-weighting approaches in automatic text retrieval10.1016/0306-4573(88)90021-0 ↗
44. MULAN: A Java library for multi-label learning