journal article·2024-12-09Open access

A Comprehensive Approach to Bias Mitigation for Sentiment Analysis of Social Media Data

Jothi Prakash Venugopal, Arul Antran Vijay Subramanian, Gopikrishnan Sundaram, Marco Rivera, Patrick Wheeler

Applied Sciences· Volume 14· Issue 23

Abstract

Sentiment analysis is a vital component of natural language processing (NLP), enabling the classification of text into positive, negative, or neutral sentiments. It is widely used in customer feedback analysis and social media monitoring but faces a significant challenge: bias. Biases, often introduced through imbalanced training datasets, can distort model predictions and result in unfair outcomes. To address this, we propose a bias-aware sentiment analysis framework leveraging Bias-BERT (Bidirectional Encoder Representations from Transformers), a customized classifier designed to balance accuracy and fairness. Our approach begins with adapting the Jigsaw Unintended Bias in Toxicity Classification dataset by converting toxicity scores into sentiment labels, making it suitable for sentiment analysis. This process includes data preparation steps like cleaning, tokenization, and feature extraction, all aimed at reducing bias. At the heart of our method is a novel loss function incorporating a bias-aware term based on the Kullback–Leibler (KL) divergence. This term guides the model toward fair predictions by penalizing biased outputs while maintaining robust classification performance. Ethical considerations are integral to our framework, ensuring the responsible deployment of AI models. This methodology highlights a pathway to equitable sentiment analysis by actively mitigating dataset biases and promoting fairness in NLP applications.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Sentiment Analysis and Opinion Mining · Spam and Phishing Detection · Topic Modeling

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Information Technology, Karpagam College of Engineering, Myleripalayam Village, Coimbatore 641032, Tamil Nadu, India

Department of Computer Science and Engineering, Karpagam College of Engineering, Myleripalayam Village, Coimbatore 641032, Tamil Nadu, India

School of Computer Science and Engineering, VIT-AP University, Amaravati 522237, Andhra Pradesh, India

Power Electronics, Machines and Control (PEMC) Research Institute, University of Nottingham, 15 Triumph Rd, Lenton, Nottingham NG7 2RD, UK

References (60)

Show source-provided references
1. Aggarwal, C.C. (2022). Opinion Mining and Sentiment Analysis. Machine Learning for Text, Springer International Publishing.10.1007/978-3-030-96623-2_15 ↗
2. Chen, Y., Zhou, X., Bai, X., Liu, B., Chen, F., Chang, L., and Liu, H. (2024). A systematic review and meta-analysis of the effectiveness of social support on turnover intention in clinical nurses. Front. Public Health, 12.10.3389/fpubh.2024.1393024 ↗
3. Digital Health Interventions and Patient Safety in Abdominal Surgery10.1001/jamanetworkopen.2024.8555 ↗
4. Pillai, S.E.V.S., Vallabhaneni, R., Pareek, P.K., and Dontu, S. (2024, January 15–16). The People Moods Analysing Using Tweets Data on Primary Things with the Help of Advanced Techniques. Proceedings of the 2024 IEEE International Conference on Distributed Computing and Optimization Techniques (ICDCOT), Bengaluru, India.10.1109/ICDCOT61034.2024.10516073 ↗
5. Ma, P., and Johnson, N. (2024, August 11). Examine the Association Between Particulate Matter Exposure and Symptoms. Available online: https://osf.io/x9672/resources.
6. Ferrara, E. (2024). Fairness and Bias in Artificial Intelligence: A Brief Survey of Sources, Impacts, and Mitigation Strategies. Sci, 6.10.2196/preprints.48399 ↗
7. Zhang, B.H., Lemoine, B., and Mitchell, M. (2018, January 2–3). Mitigating unwanted biases with adversarial learning. Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, New Orleans, LA, USA.10.1145/3278721.3278779 ↗
8. Research on opinion polarization by big data analytics capabilities in online social networks10.1016/j.techsoc.2022.101902 ↗
9. Unpacking government social media messaging strategies during the COVID-19 pandemic in China10.1002/poi3.282 ↗
10. Developing human/AI interactions for chat-based customer services: Lessons learned from the Norwegian government10.1080/0960085X.2022.2096490 ↗
11. Use of sentiment analysis in social media campaign design and analysis10.18137/cardiometry.2022.22.351363 ↗
12. Machine learning-based proactive social-sensor service for mental health monitoring using Twitter data
13. Farha, I.A., Wilson, S., Oprea, S., and Magdy, W. (2022, January 7–11). Sarcasm Detection is Way Too Easy! An Empirical Comparison of Human and Machine Sarcasm Detection. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates.
14. Khoo, L.S., Bay, J.Q., Yap, M.L.K., Lim, M.K., Chong, C.Y., Yang, Z., and Lo, D. (2023, January 21–24). Exploring and Repairing Gender Fairness Violations in Word Embedding-based Sentiment Analysis Model through Adversarial Patches. Proceedings of the 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), Macao, China.
15. Hong, M.H., Marsh, L.A., Feuston, J.L., Ruppert, J., Brubaker, J.R., and Szafir, D.A. (November, January 29). Scholastic: Graphical Human-AI Collaboration for Inductive and Interpretive Text Analysis. Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, Bend, OR, USA.
16. Mamun, M.H.A., Keikhosrokiani, P., Asl, M.P., Anuar, N.A.N., Hadi, N.H.A., and Humida, T. (2022). Sentiment Analysis of the Harry Potter Series Using a Lexicon-Based Approach. Advances in Sentiment Analysis and Natural Language Processing, IGI Global.10.4018/978-1-7998-9594-7.ch011 ↗
17. A lexicon-based approach to examine depression detection in social media: The case of Twitter and university community10.1057/s41599-022-01313-2 ↗
18. Ainapure, B.S., Pise, R.N., Reddy, P., Appasani, B., Srinivasulu, A., Khan, M.S., and Bizon, N. (2023). Sentiment Analysis of COVID-19 Tweets Using Deep Learning and Lexicon-Based Approaches. Sustainability, 15.10.3390/su15032573 ↗
19. Political security threat prediction framework using hybrid lexicon-based approach and machine learning technique10.1109/ACCESS.2023.3246162 ↗
20. A lexicon-based approach for sentiment analysis of multimodal content in tweets10.1007/s11042-023-14411-3 ↗
21. AlBadani, B., Shi, R., and Dong, J. (2022). A novel machine learning approach for sentiment analysis on Twitter incorporating the Universal Language Model Fine-Tuning and SVM. Appl. Syst. Innov., 5.10.3390/asi5010013 ↗
22. A sentiment analysis model of Agritech startup on Facebook comments using naive Bayes classifier10.11591/ijece.v12i3.pp2829-2838 ↗
23. An empirical comparison of machine learning methods for text-based sentiment analysis of online consumer reviews10.1016/j.ijresmar.2021.10.011 ↗
24. A novel unsupervised ensemble framework using concept-based linguistic methods and machine learning for Twitter sentiment analysis10.1016/j.patrec.2022.04.004 ↗
25. Machine learning sentiment analysis, COVID-19 news and stock market reactions10.1016/j.ribaf.2023.101881 ↗
26. Quantifying the effect of eWOM embedded consumer perceptions on sales: An integrated aspect-level sentiment analysis and panel data modeling approach10.1016/j.jbusres.2021.08.060 ↗
27. A survey on deep learning for textual emotion analysis in social networks10.1016/j.dcan.2021.10.003 ↗
28. A mixed approach of deep learning method and rule-based method to improve aspect level sentiment analysis10.1016/j.aci.2019.02.002 ↗
29. ACR-SA: Attention-based deep model through two-channel CNN and Bi-RNN for sentiment analysis10.7717/peerj-cs.877 ↗
30. Khan, L., Amjad, A., Ashraf, N., and Chang, H.T. (2022). Multi-class sentiment analysis of Urdu text using multilingual BERT. Sci. Rep., 12.10.1038/s41598-022-09381-9 ↗
31. An optimal deep learning-based LSTM for stock price prediction using twitter sentiment analysis10.1007/s10489-022-03175-2 ↗
32. Sentiment analysis using machine learning: Progress in the machine intelligence for data science
33. Transformer-based deep learning models for the sentiment analysis of social media data10.1016/j.array.2022.100157 ↗
34. Understanding the Perceptions of Healthcare Researchers Regarding ChatGPT: A Study Based on Bidirectional Encoder Representation from Transformers (BERT) Sentiment Analysis and Topic Modeling10.1007/s10439-023-03222-0 ↗
35. Sentiment spin: Attacking financial sentiment with GPT-310.1016/j.frl.2023.103957 ↗
36. RoBERTa-LSTM: A Hybrid Model for Sentiment Analysis With Transformer and Recurrent Neural Network10.1109/ACCESS.2022.3152828 ↗
37. Azhar, N., and Latif, S. (2022, January 28–29). Roman Urdu Sentiment Analysis Using Pre-trained DistilBERT and XLNet. Proceedings of the 2022 IEEE Fifth International Conference of Women in Data Science at Prince Sultan University (WiDS PSU), Riyadh, Saudi Arabia.10.1109/WiDS-PSU54548.2022.00027 ↗
38. Anoop, K., Gangan, M.P., Deepak, P., and Lajish, V.L. (2022). Towards an Enhanced Understanding of Bias in Pre-trained Neural Language Models: A Survey with Special Emphasis on Affective Bias. Advances in Neural Networks and Machine Learning, Springer.10.1007/978-981-19-4453-6_2 ↗
39. Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysis10.1162/coli_a_00433 ↗
40. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope10.1016/j.iotcps.2023.04.003 ↗
41. The Biases of Pre-Trained Language Models: An Empirical Study on Prompt-Based Sentiment Analysis and Emotion Detection10.1109/TAFFC.2022.3204972 ↗
42. Hartung, K., Herygers, A., Kurlekar, S.V., Zakaria, K., Volkan, T., Gröttrup, S., and Georges, M. (2023). Measuring Sentiment Bias in Machine Translation. Advances in Data Science and Deep Learning, Springer.10.1007/978-3-031-40498-6_8 ↗
43. Orgad, H., and Belinkov, Y. (2023, January 9–14). BLIND: Bias Removal With No Demographics. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, ON, Canada.10.18653/v1/2023.acl-long.490 ↗
44. Sun, T., Wang, W., Jing, L., Cui, Y., Song, X., and Nie, L. (2022, January 10–14). Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis. Proceedings of the 30th ACM International Conference on Multimedia, Lisbon, Portugal.10.1145/3503161.3548211 ↗
45. Addressing algorithmic bias and the perpetuation of health inequities: An AI bias aware framework10.1016/j.hlpt.2022.100702 ↗
46. Engineers, Aware! Commercial Tools Disagree on Social Media Sentiment: Analyzing the Sentiment Bias of Four Major Tools
47. Bias-Aware Design for Informed Decisions: Raising Awareness of Self-Selection Bias in User Ratings and Reviews10.1145/3555597 ↗
48. Boonprakong, N., Chen, X., Davey, C., Tag, B., and Dingler, T. (2023, January 23–28). Bias-Aware Systems: Exploring Indicators for the Occurrences of Cognitive Biases when Facing Different Opinions. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany.10.1145/3544548.3580917 ↗
49. Taeb, M., Torres, Y., Chi, H., and Bernadin, S. (2022, January 14–16). Investigating Gender and Racial Bias in ELECTRA. Proceedings of the 2022 IEEE International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA.10.1109/CSCI58124.2022.00027 ↗
50. Cheng, L., Mosallanezhad, A., Silva, Y.N., Hall, D.L., and Liu, H. (2022, January 11–15). Bias Mitigation for Toxicity Detection via Sequential Decisions. Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain.10.1145/3477495.3531945 ↗
51. Garlapati, A., Malisetty, N., and Narayanan, G. (2022, January 25–26). Classification of Toxicity in Comments Using NLP and LSTM. Proceedings of the IEEE 8th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India.10.1109/ICACCS54159.2022.9785067 ↗
52. Sentiment Analysis of Tweets During the COVID-19 Pandemic Using Multinomial Logistic Regression
53. Raj, C., Agarwal, A., Bharathy, G., Narayan, B., and Prasad, M. (2021). Cyberbullying detection: Hybrid models based on machine learning and natural language processing techniques. Electronics, 10.10.3390/electronics10222810 ↗
54. Comparison analysis of Bangla news articles classification using support vector machine and logistic regression10.12928/telkomnika.v21i3.23416 ↗
55. Using recurrent neural network models for early detection of heart failure onset10.1093/jamia/ocw112 ↗
56. A multi-aspect framework for explainable sentiment analysis10.1016/j.patrec.2024.01.001 ↗
57. Kaushik, D., Hovy, E., and Lipton, Z.C. (2019). Learning the difference that makes a difference with counterfactually-augmented data. arXiv.
58. Mitigating bias in artificial intelligence: Fair data generation via causal models for transparent and explainable decision-making10.1016/j.future.2024.02.023 ↗
59. Clark, C., Yatskar, M., and Zettlemoyer, L. (2019). Don’t take the easy way out: Ensemble based methods for avoiding known dataset biases. arXiv.10.18653/v1/D19-1418 ↗
60. BiasFinder: Metamorphic Test Generation to Uncover Bias for Sentiment Analysis Systems