BTCAM: Attention-Based BiLSTM for Imbalanced Classification of Political Inquiry Messages
Source abstract
Abstract
The classification of political inquiry messages is a crucial task in government affairs. However, with the increasing number of inquiry messages on platforms, it is difficult for government departments to accurately and efficiently categorize these messages solely through manual labor. Existing methods find it challenging to achieve excellent performance due to the sparsity of the features and the imbalanced data. In this paper, we propose a new framework (BTCAM) for automatically classifying political inquiry messages. Specifically, we first propose a topic-based data augmentation algorithm (TDA) to improve the diversity and quality of data. In addition, in order to enable the model to focus on the key information in the text, we propose an applicable streamlined convolutional block attention module (SCBAM), which can highlight the salient features on the channel and spatial axes. Extensive experiments show that the accuracy and recall of BTCAM reach 0.939 and 0.931, respectively, outperforming the state-of-the-art methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Text and Document Classification Technologies · Sentiment Analysis and Opinion Mining
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science, South China Normal University, Guangzhou 510631, China
School of Artificial Intelligence, South China Normal University, Foshan 528225, China
Funding
Artificial Intelligence Robot Industry College of South China Normal University (Guangdong Provincial Model Industry College) · Scientific Research Innovation Project of Graduate School of South China Normal University
References (42)
Show source-provided references
1. A corpus analysis of electronic petitions for improving the responsiveness of public services: Focusing on Busan petition10.20484/klog.21.1.17 ↗
2. Automatic classification of citizen requests for transportation using deep learning: Case study from Boston city10.1016/j.ipm.2020.102410 ↗
3. Dalianis, H., Sjöbergh, J., and Sneiders, E. (2011). Comparing manual text patterns and machine learning for classification of e-mails for automatic answering by a government agency. Computational Linguistics and Intelligent Text Processing, Springer.10.1007/978-3-642-19437-5_19 ↗
4. An integrated artificial intelligence framework for public management10.1080/14719037.2018.1549268 ↗
5. An analysis of civil traffic complaints using text mining
6. Content analysis of e-petitions with topic modeling: How to train and evaluate LDA models?10.1016/j.ipm.2018.05.006 ↗
7. A pattern analysis of bus civil complaint in Busan city using the text network analysis10.32956/kaoca.2018.16.2.19 ↗
8. A multi-class SVM classification system based on learning methods from indistinguishable Chinese official documents10.1016/j.eswa.2011.08.176 ↗
9. Dieng, A.B., Wang, C., Gao, J., and Paisley, J. (2016). Topicrnn: A recurrent neural network with long-range semantic dependency. arXiv.
10. A lightweight CORONA-NET for COVID-19 detection in X-ray images10.1016/j.eswa.2023.120023 ↗
11. Prediction of daily average seawater temperature using data-driven and deep learning algorithms10.1007/s00521-023-09010-0 ↗
12. A neural probabilistic bounded confidence model for opinion dynamics on social networks10.1016/j.eswa.2024.123315 ↗
13. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019, January 2–7). Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA. Long and Short Papers.
14. A spatial and temporal sentiment analysis approach applied to Twitter microtexts
15. Recurrent convolutional neural networks for text classification
16. The automatic classification and matching of informal texts on political media integration platform
17. Research on automatic forwarding method of government website mailbox based on text classification
18. Research on government message text classification based on natural Language Processing
19. An empirical survey of data augmentation for limited data learning in NLP10.1162/tacl_a_00542 ↗
20. Feng, S.Y., Gangal, V., Wei, J., Chandar, S., Vosoughi, S., Mitamura, T., and Hovy, E. (2021). A survey of data augmentation approaches for NLP. arXiv.10.18653/v1/2021.findings-acl.84 ↗
21. Zhang, L., Yang, Z., and Yang, D. (2022). TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding. arXiv.10.18653/v1/2022.naacl-main.385 ↗
22. Hariharan, B., and Girshick, R. (2017, January 22–29). Low-shot visual recognition by shrinking and hallucinating features. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.328 ↗
23. Chen, J., Yang, Z., and Yang, D. (2020). Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification. arXiv.10.18653/v1/2020.acl-main.194 ↗
24. Sennrich, R., Haddow, B., and Birch, A. (2015). Improving neural machine translation models with monolingual data. arXiv.10.18653/v1/P16-1009 ↗
25. Yang, Y., Malaviya, C., Fernandez, J., Swayamdipta, S., Bras, R.L., Wang, J.P., Bhagavatula, C., Choi, Y., and Downey, D. (2020). Generative data augmentation for commonsense reasoning. arXiv.10.18653/v1/2020.findings-emnlp.90 ↗
26. Wei, J., and Zou, K. (2019). Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv.10.18653/v1/D19-1670 ↗
27. Karimi, A., Rossi, L., and Prati, A. (2021). Aeda: An easier data augmentation technique for text classification. arXiv.10.18653/v1/2021.findings-emnlp.234 ↗
28. Fu, Y., Xu, D., He, K., Li, H., and Zhang, T. (2022, January 7–9). Image Inpainting Based on Edge Features and Attention Mechanism. Proceedings of the 2022 5th International Conference on Image and Graphics Processing (ICIGP), Beijing, China.10.1145/3512388.3512398 ↗
29. Sentiment classification of microblog: A framework based on BERT and CNN with attention mechanism10.1016/j.compeleceng.2022.108032 ↗
30. Optimizing the Online Learners’ Verbal Intention Classification Efficiency Based on the Multi-Head Attention Mechanism Algorithm10.1142/S0129054122420114 ↗
31. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
32. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
33. Indexing by latent semantic analysis10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9 ↗
34. Hofmann, T. (2013). Probabilistic latent semantic analysis. arXiv.
35. Latent dirichlet allocation
36. Khan, S., Durrani, S., Shahab, M.B., Johnson, S.J., and Camtepe, S. (2022). Joint User and Data Detection in Grant-Free NOMA with Attention-based BiLSTM Network. arXiv.10.1109/OJCOMS.2023.3292820 ↗
37. Li, J., and Sun, M. (2007, January 28–30). Scalable term selection for text categorization. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), Prague, Czech Republic.
38. Kim, Y. (2014). Convolutional Neural Networks for Sentence Classification. arXiv.10.3115/v1/D14-1181 ↗
39. Liu, P., Qiu, X., and Huang, X. (2016). Recurrent neural network for text classification with multi-task learning. arXiv.
40. Joulin, A., Grave, E., Bojanowski, P., and Mikolov, T. (2016). Bag of tricks for efficient text classification. arXiv.10.18653/v1/E17-2068 ↗
41. Zhou, P., Shi, W., Tian, J., Qi, Z., Li, B., Hao, H., and Xu, B. (2016, January 7–12). Attention-based bidirectional long short-term memory networks for relation classification. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-2034 ↗
42. Salerno, M., Sargeni, F., and Bonaiuto, V. (May, January 30). DPCNN: A modular chip for large CNN arrays. Proceedings of the ISCAS’95-International Symposium on Circuits and Systems, Seattle, WA, USA.