journal article·2022-12-27Open access

DPG-LSTM: An Enhanced LSTM Framework for Sentiment Analysis in Social Media Text Based on Dependency Parsing and GCN

Zeyu Yin, Jinsong Shao, Muhammad Jawad Hussain, Yajie Hao, Yu Chen, Xuefeng Zhang, Li Wang

Applied Sciences· Volume 13· Issue 1

Abstract

Sentiment analysis based on social media text is found to be essential for multiple applications such as project design, measuring customer satisfaction, and monitoring brand reputation. Deep learning models that automatically learn semantic and syntactic information have recently proved effective in sentiment analysis. Despite earlier studies’ good performance, these methods lack syntactic information to guide feature development for contextual semantic linkages in social media text. In this paper, we introduce an enhanced LSTM-based on dependency parsing and a graph convolutional network (DPG-LSTM) for sentiment analysis. Our research aims to investigate the importance of syntactic information in the task of social media emotional processing. To fully utilize the semantic information of social media, we adopt a hybrid attention mechanism that combines dependency parsing to capture semantic contextual information. The hybrid attention mechanism redistributes higher attention scores to words with higher dependencies generated by dependency parsing. To validate the performance of the DPG-LSTM from different perspectives, experiments have been conducted on three tweet sentiment classification datasets, sentiment140, airline reviews, and self-driving car reviews with 1,604,510 tweets. The experimental results show that the proposed DPG-LSTM model outperforms the state-of-the-art model by 2.1% recall scores, 1.4% precision scores, and 1.8% F1 scores on sentiment140.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Sentiment Analysis and Opinion Mining · Topic Modeling · Text and Document Classification Technologies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Information Science and Technology, Nantong University, Nantong 226007, China

Research Center for Intelligence Information Technology, Nantong University, Nantong 226007, China

Funding

National Science Foundation of China · Project of Nantong Health Commission · Science and Technology Project of Nantong City

References (45)

Show source-provided references
1. The Effect of Social Media User Behaviors on Security and Privacy Threats10.1109/ACCESS.2022.3177652 ↗
2. Estimating sentence types in computer related new product bulletins using a decision tree10.1016/j.ins.2004.02.004 ↗
3. National Research Council (2013). Frontiers in Massive Data Analysis, National Academies Press.
4. Ju, S., and Li, S. (2013). Active learning on sentiment classification by selecting both words and documents. Workshop on Chinese Lexical Semantics, Springer.10.1007/978-3-642-36337-5_6 ↗
5. Liu, L., Nie, X., and Wang, H. (2012, January 16–18). Toward a fuzzy domain sentiment ontology tree for sentiment analysis. Proceedings of the International Congress on Image and Signal Processing, Chongqing, China.10.1109/CISP.2012.6469930 ↗
6. Social context in sentiment analysis: Formal definition, overview of current trends and framework for comparison10.1016/j.inffus.2019.05.003 ↗
7. A comprehensive survey on sentiment analysis: Approaches, challenges, and trends10.1016/j.knosys.2021.107134 ↗
8. Zhao, Z., Rao, G., and Feng, Z. (2017, January 7–9). DFDS: A Domain-Independent Framework for Document-Level Sentiment Analysis Based on RST. Proceedings of the Asia-Pacific Web and Web-Age Information Management Joint Conference on Web and Big Data, Beijing, China.10.1007/978-3-319-63579-8_23 ↗
9. Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN10.1016/j.eswa.2016.10.065 ↗
10. Tang, D., and Zhang, M. (2008). Deep Learning in Sentiment Analysis. Deep Learning in Natural Language Processing, Springer.
11. Zhang, C., Li, Q., and Song, D. (2019, January 3–7). Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China.10.18653/v1/D19-1464 ↗
12. Mukherjee, S., Malu, A., Ar, B., and Bhattacharyya, P. (November, January 29). Feature specific sentiment analysis for product reviews. Proceedings of the International Conference on Intelligent Text Processing and Computational Linguistics, Maui, HI, USA.
13. Kponyo, J.J., Kuang, Y., Zhang, E., and Domenic, K. (2013, January 26–28). VANET cluster-on-demand minimum spanning tree (MST) prim clustering algorithm. Proceedings of the International Conference on Computational Problem-Solving, Jiuzhai, China.10.1109/ICCPS.2013.6893585 ↗
14. Kipf, T.N., and Welling, M. (2019). Semi-supervised classification with graph convolutional networks. arXiv.
15. Sun, K., Zhang, R., Mensah, S., Mao, Y., and Liu, X. (2019, January 3–7). Aspect-level sentiment analysis via convolution over dependency tree. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Online.10.18653/v1/D19-1569 ↗
16. Aspect-based sentiment analysis with new target representation and dependency attention10.1109/TAFFC.2019.2945028 ↗
17. Investigation in the influences of public opinion indicators on vegetable prices by corpora construction and WeChat article analysis10.1016/j.future.2019.07.016 ↗
18. A natural language processing pipeline of chinese free-text radiology reports for liver cancer diagnosis10.1109/ACCESS.2020.3020138 ↗
19. Knowledge-based sentiment analysis and visualization on social networks10.1007/s00354-020-00103-1 ↗
20. Agarwal, B. (2022). Financial sentiment analysis model utilizing knowledge-base and domain-specific representation. Multimed. Tools Appl., 1–22.10.1007/s11042-022-12181-y ↗
21. Sentic LSTM: A Hybrid Network for Targeted Aspect-Based Sentiment Analysis10.1007/s12559-018-9549-x ↗
22. Ma, Y., Peng, H., and Cambria, E. (2018, January 2–7). Targeted Aspect-Based Sentiment Analysis via Embedding Commonsense Knowledge into an Attentive LSTM. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12048 ↗
23. Multi-head self-attention transformation networks for aspect-based sentiment analysis10.1109/ACCESS.2021.3049294 ↗
24. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
25. Depression risk prediction for chinese microblogs via deep-learning methods: Content analysis10.2196/17958 ↗
26. Xlnet: Generalized autoregressive pretraining for language understanding
27. Joulin, A., Grave, E., Bojanowski, P., and Mikolov, T. (2016). Bag of tricks for efficient text classification. arXiv.10.18653/v1/E17-2068 ↗
28. Cross-Cultural Polarity and Emotion Detection Using Sentiment Analysis and Deep Learning on COVID-19 Related Tweets10.1109/ACCESS.2020.3027350 ↗
29. AlBadani, B., Shi, R., and Dong, J. (2022). A Novel Machine Learning Approach for Sentiment Analysis on Twitter Incorporating the Universal Language Model Fine-Tuning and SVM. Appl. Syst. Innov., 5.10.3390/asi5010013 ↗
30. Zhang, M., and Qian, T. (2020, January 16–20). Convolution over hierarchical syntactic and lexical graphs for aspect level sentiment analysis. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Online.10.18653/v1/2020.emnlp-main.286 ↗
31. Ghosal, D., Hazarika, D., Roy, A., Majumder, N., Mihalcea, R., and Poria, S. (2020, January 5–10). KinGDOM: Knowledge-Guided DOMain Adaptation for Sentiment Analysis. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.292 ↗
32. Speer, R., and Havasi, C. (2013). ConceptNet 5: A large semantic network for relational knowledge. The People’s Web Meets NLP, Association for Computational Linguistics.10.1007/978-3-642-35085-6_6 ↗
33. Chen, X., Sun, C., Wang, J., Li, S., Si, L., Zhang, M., and Zhou, G. (2020, January 5–10). Aspect sentiment classification with document-level sentiment preference modeling. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.338 ↗
34. Multi-level graph neural network for text sentiment analysis10.1016/j.compeleceng.2021.107096 ↗
35. Phrase dependency relational graph attention network for Aspect-based Sentiment Analysis10.1016/j.knosys.2021.107736 ↗
36. A semantic and syntactic enhanced neural model for financial sentiment analysis10.1016/j.ipm.2022.102943 ↗
37. Sentiment analysis based on improved pre-trained word embeddings10.1016/j.eswa.2018.08.044 ↗
38. Zou, H., Tang, X., Xie, B., and Liu, B. (2015, January 7–9). Sentiment Classification Using Machine Learning Techniques with Syntax Features. Proceedings of the International Conference on Computational Science and Computational Intelligence, Las Vegas, NV, USA.10.1109/CSCI.2015.44 ↗
39. Towards a real-time processing framework based on improved distributed recurrent neural network variants with fastText for social big data analytics10.1016/j.ipm.2019.102122 ↗
40. Bifet, A., and Frank, E. (2010). Sentiment Knowledge Discovery in Twitter Streaming Data. International Conference on Discovery Science, Springer.10.1007/978-3-642-16184-1_1 ↗
41. Chen, L.C., Barron, J.T., Papandreou, G., Murphy, K., and Yuille, A.L. (July, January 26). Semantic image segmentation with task-specific edge detection using CNNs and a discriminatively trained domain transform. Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA.
42. Bidirectional LSTM with attention mechanism and convolutional layer for text classification10.1016/j.neucom.2019.01.078 ↗
43. ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis10.1016/j.future.2020.08.005 ↗
44. Stacked DeBERT: All attention in incomplete data for text classification10.1016/j.neunet.2020.12.018 ↗
45. Kamyab, M., Liu, G., and Adjeisah, M. (2021). Attention-based CNN and Bi-LSTM model based on TF-IDF and glove word embedding for sentiment analysis. Appl. Sci., 11.10.3390/app112311255 ↗