LCF: A Local Context Focus Mechanism for Aspect-Based Sentiment Classification
Source abstract
Abstract
Aspect-based sentiment classification (ABSC) aims to predict sentiment polarities of different aspects within sentences or documents. Many previous studies have been conducted to solve this problem, but previous works fail to notice the correlation between the aspect’s sentiment polarity and the local context. In this paper, a Local Context Focus (LCF) mechanism is proposed for aspect-based sentiment classification based on Multi-head Self-Attention (MHSA). This mechanism is called LCF design, and utilizes the Context features Dynamic Mask (CDM) and Context Features Dynamic Weighted (CDW) layers to pay more attention to the local context words. Moreover, a BERT-shared layer is adopted to LCF design to capture internal long-term dependencies of local context and global context. Experiments are conducted on three common ABSC datasets: the laptop and restaurant datasets of SemEval-2014 and the ACL twitter dataset. Experimental results demonstrate that the LCF baseline model achieves considerable performance. In addition, we conduct ablation experiments to prove the significance and effectiveness of LCF design. Especially, by incorporating with BERT-shared layer, the LCF-BERT model refreshes state-of-the-art performance on all three benchmark datasets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Sentiment Analysis and Opinion Mining · Text and Document Classification Technologies · Topic Modeling
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Software, South China Normal University, Foshan 528225, China
School of Computer, South China Normal University, Guangzhou 510631, China
References (34)
Show source-provided references
1. Maria Pontiki, D.G., John Pavlopoulos, H.P., and Ion Androutsopoulos, S.M. (2014, January 23–24). Semeval-2014 task 4: SemEval-2014 Task 4: Aspect Based Sentiment Analysis. Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), Dublin, Ireland.10.3115/v1/S14-2004 ↗
2. Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Mohammad, A.S., Al-Ayyoub, M., Zhao, Y., Qin, B., and De Clercq, O. (2016, January 16–17). Semeval-2016 task 5: Aspect based sentiment analysis. Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego, CA, USA.10.18653/v1/S16-1002 ↗
3. Opinion mining and sentiment analysis10.1561/1500000011 ↗
4. Sentiment analysis and opinion mining
5. Kim, H., and Jeong, Y.S. (2019). Sentiment Classification Using Convolutional Neural Networks. Appl. Sci., 9.10.3390/app9112347 ↗
6. Agarwal, A., Xie, B., Vovsha, I., Rambow, O., and Passonneau, R. (2011, January 23). Sentiment analysis of twitter data. Proceedings of the Workshop on Language in Social Media (LSM 2011), Portland, OR, USA.
7. Kanayama, H., and Nasukawa, T. (2006, January 22–23). Fully automatic lexicon expansion for domain-oriented sentiment analysis. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Sydney, Australia.10.3115/1610075.1610125 ↗
8. McDonald, R., Hannan, K., Neylon, T., Wells, M., and Reynar, J. (2007, January 25–27). Structured models for fine-to-coarse sentiment analysis. Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech Republic.
9. Recognizing contextual polarity: An exploration of features for phrase-level sentiment analysis10.1162/coli.08-012-R1-06-90 ↗
10. Lexicon-based methods for sentiment analysis10.1162/COLI_a_00049 ↗
11. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, Inc.
12. Wang, Y., Huang, M., and Zhao, L. (2016, January 1–5). Attention-based LSTM for aspect-level sentiment classification. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, TX, USA.10.18653/v1/D16-1058 ↗
13. Ma, D., Li, S., Zhang, X., and Wang, H. (2017, January 19–25). Interactive attention networks for aspect-level sentiment classification. Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Australia.10.24963/ijcai.2017/568 ↗
14. Li, X., Bing, L., Lam, W., and Shi, B. (2018, January 15–20). Transformation Networks for Target-Oriented Sentiment Classification. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia.10.18653/v1/P18-1087 ↗
15. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: pretraining of deep bidirectional transformers for language understanding. arXiv.
16. Kiritchenko, S., Zhu, X., Cherry, C., and Mohammad, S. (2014, January 23–24). NRC-Canada-2014: Detecting aspects and sentiment in customer reviews. Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), Dublin, Ireland.10.3115/v1/S14-2076 ↗
17. Vo, D.T., and Zhang, Y. (2015, January 25–31). Target-dependent twitter sentiment classification with rich automatic features. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina.
18. A neural probabilistic language model
19. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, Inc.
20. Le, Q., and Mikolov, T. (2014, January 21–26). Distributed representations of sentences and documents. Proceedings of the International Conference on Machine Learning, Beijing, China.
21. Pennington, J., Socher, R., and Manning, C. (2014, January 25–29). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1162 ↗
22. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
23. Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, Inc.
24. Tang, D., Qin, B., Feng, X., and Liu, T. (2016, January 11–17). Effective LSTMs for Target-Dependent Sentiment Classification. Proceedings of the COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan.
25. Chen, P., Sun, Z., Bing, L., and Yang, W. (2017, January 7–11). Recurrent attention network on memory for aspect sentiment analysis. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.10.18653/v1/D17-1047 ↗
26. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.10.3115/v1/D14-1179 ↗
27. Fan, F., Feng, Y., and Zhao, D. (November, January 31). multigrained attention network for aspect-level sentiment classification. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1380 ↗
28. Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018, January 1–6). Deep contextualized word representations. Proceedings of the NAACL-HLT, New Orleans, LA, USA.10.18653/v1/N18-1202 ↗
29. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2018, June 11). Improving Language Understanding by Generative Pretraining. Available online: https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/languageunsupervised/languageunderstandingpaper.pdf.
30. Xu, H., Liu, B., Shu, L., and Philip, S.Y. (2019, January 2–7). BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.
31. Song, Y., Wang, J., Jiang, T., Liu, Z., and Rao, Y. (2019). Attentional Encoder Network for Targeted Sentiment Classification. arXiv.10.1007/978-3-030-30490-4_9 ↗
32. Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., and Xu, K. (2014, January 23–25). Adaptive recursive neural network for target-dependent twitter sentiment classification. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Baltimore, MD, USA.10.3115/v1/P14-2009 ↗
33. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
34. Tang, D., Qin, B., and Liu, T. (2016, January 1–5). Aspect Level Sentiment Classification with Deep Memory Network. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, TX, USA.10.18653/v1/D16-1021 ↗