Interpretability in Sentiment Analysis: A Self-Supervised Approach to Sentiment Cue Extraction
Source abstract
Abstract
In this paper, we present a novel self-supervised framework for Sentiment Cue Extraction (SCE) aimed at enhancing the interpretability of text sentiment analysis models. Our approach leverages self-supervised learning to identify and highlight key textual elements that significantly influence sentiment classification decisions. Central to our framework is the development of an innovative Mask Sequence Interpretation Score (MSIS), a bespoke metric designed to assess the relevance and coherence of identified sentiment cues within binary text classification tasks. By employing Monte Carlo Sampling techniques optimized for computational efficiency, our framework demonstrates exceptional effectiveness in processing large-scale text data across diverse datasets, including English and Chinese, thus proving its versatility and scalability. The effectiveness of our approach is validated through extensive experiments on several benchmark datasets, including SST-2, IMDb, Yelp, and ChnSentiCorp. The results indicate a substantial improvement in the interpretability of the sentiment analysis models without compromising their predictive accuracy. Furthermore, our method stands out for its global interpretability, offering an efficient solution for analyzing new data compared to traditional techniques focused on local explanations.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Explainable Artificial Intelligence (XAI) · Sentiment Analysis and Opinion Mining
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of Trustworthy Distributed Computing and Service (BUPT), Ministry of Education, Beijing University of Posts and Telecommunications, Beijing 100876, China
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing 100876, China
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
Institute of Scientific and Technical Information of China, Beijing 100038, China
Institute of Medical Information, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing 100020, China
Funding
National Natural Science Foundation of China
References (48)
Show source-provided references
1. Liu, B. (2022). Sentiment Analysis and Opinion Mining, Springer Nature.
2. Opinion mining and sentiment analysis10.1561/1500000011 ↗
3. A survey on sentiment analysis methods, applications, and challenges10.1007/s10462-022-10144-1 ↗
4. Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., and Kagal, L. (2018, January 1–3). Explaining explanations: An overview of interpretability of machine learning. Proceedings of the 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), Turin, Italy.10.1109/DSAA.2018.00018 ↗
5. Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13–17). “Why should i trust you?” Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939778 ↗
6. A novel ensemble learning approach for stock market prediction based on sentiment analysis and the sliding window method10.1109/TCSS.2022.3182375 ↗
7. McCarthy, S., and Alaghband, G. (2023). Enhancing Financial Market Analysis and Prediction with Emotion Corpora and News Co-Occurrence Network. J. Risk Financ. Manag., 16.10.3390/jrfm16040226 ↗
8. Bharti, S.K., Tratiya, P., and Gupta, R.K. (2022, January 15–17). Stock Market Price Prediction through News Sentiment Analysis & Ensemble Learning. Proceedings of the 2022 IEEE 2nd International Symposium on Sustainable Energy, Signal Processing and Cyber Security (iSSSC), Odisha, India.10.1109/iSSSC56467.2022.10051623 ↗
9. Use of sentiment analysis for capturing patient experience from free-text comments posted online
10. From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI10.1145/3583558 ↗
11. Post-hoc interpretability for neural nlp: A survey10.1145/3546577 ↗
12. Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities10.1016/j.knosys.2023.110273 ↗
13. A survey of sentiment analysis in social media10.1007/s10115-018-1236-4 ↗
14. Deep learning for sentiment analysis: A survey10.1002/widm.1253 ↗
15. Liu, Y. (2019). Fine-tune BERT for extractive summarization. arXiv.
16. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
17. Language models are few-shot learners
18. Maas, A., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., and Potts, C. (2011, January 19–24). Learning word vectors for sentiment analysis. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA.
19. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C.D., Ng, A.Y., and Potts, C. (2013, January 18–21). Recursive Deep Models for Semantic Compositionality over a Sentiment Treebank. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA.
20. Kaur, R., and Kautish, S. (2022). Research Anthology on Implementing Sentiment Analysis Across Multiple Disciplines, IGI Global.10.4018/978-1-6684-6303-1.ch098 ↗
21. Self-supervised learning: Generative or contrastive
22. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., and Le, Q.V. (2019, January 8–14). XLNet: Generalized autoregressive pretraining for language understanding. Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, Vancouver, BC, Canada.
23. Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L. (2020, January 5–10). BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.703 ↗
24. Exploring the limits of transfer learning with a unified text-to-text transformer
25. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2024, March 19). Improving Language Understanding by Generative Pre-Training. Available online: https://www.mikecaptain.com/resources/pdf/GPT-1.pdf.
26. Language models are unsupervised multitask learners
27. Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., and Hu, X. (ACM Trans. Knowl. Discov. Data, 2023). Harnessing the power of llms in practice: A survey on chatgpt and beyond, ACM Trans. Knowl. Discov. Data, epub ahead of print.10.1145/3649506 ↗
28. Tian, S., Jin, Q., Yeganova, L., Lai, P.T., Zhu, Q., Chen, X., Yang, Y., Chen, Q., Kim, W., and Comeau, D.C. (2024). Opportunities and challenges for ChatGPT and large language models in biomedicine and health. Brief. Bioinform., 25.10.1093/bib/bbad493 ↗
29. Disease concept-embedding based on the self-supervised method for medical information extraction from electronic health records and disease retrieval: Algorithm development and validation study
30. Feldman, R., Rosenfled, B., Soderland, S., and Etzioni, O. (2006, January 27–29). Self-supervised relation extraction from the web. Proceedings of the Foundations of Intelligent Systems: 16th International Symposium, ISMIS 2006, Bari, Italy.10.1007/11875604_84 ↗
31. Towards A Rigorous Science of Interpretable Machine Learning
32. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
33. Wheeler, J.M., Cohen, A.S., and Wang, S. (2023). A Comparison of Latent Semantic Analysis and Latent Dirichlet Allocation in Educational Measurement. J. Educ. Behav. Stat., 10769986231209446.10.3102/10769986231209446 ↗
34. Bilevel Topic Model-Based Multitask Learning for Constructed-Responses Multidimensional Automated Scoring and Interpretation10.1111/emip.12550 ↗
35. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead10.1038/s42256-019-0048-x ↗
36. Hammersley, J. (2013). Monte Carlo Methods, Springer Science & Business Media.
37. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
38. Latent dirichlet allocation
39. Betancourt, M. (2017). A conceptual introduction to Hamiltonian Monte Carlo. arXiv.10.3150/16-BEJ810 ↗
40. Monte Carlo sampling methods
41. Pennington, J., Socher, R., and Manning, C.D. (2014, January 25–29). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1162 ↗
42. Bridle, J.S. (1990). Neurocomputing: Algorithms, Architectures and Applications, Springer.
43. Sundararajan, M., Taly, A., and Yan, Q. (2017, January 6–11). Axiomatic attribution for deep networks. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia.
44. Zeiler, M.D., and Fergus, R. (2014, January 6–12). Visualizing and understanding convolutional networks. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland. Proceedings, Part I 13.10.1007/978-3-319-10590-1_53 ↗
45. Polynomial calculation of the Shapley value based on sampling10.1016/j.cor.2008.04.004 ↗
46. Lundberg, S.M., and Lee, S.I. (2017, January 4–9). A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
47. Adel, H., and Shi, S. (2021, January 7–11). Thermostat: A Large Collection of NLP Model Explanations and Analysis Tools. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Virtual Event.
48. Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2019). ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. arXiv.