GenAI-Powered Framework for Reliable Sentiment Labeling in Drug Safety Monitoring
Source abstract
Abstract
The analysis of medical data presents an opportunity for healthcare systems to support decision-making and improve patient outcomes. In this context, the automated analysis of user-generated drug reviews offers a promising approach for monitoring medication safety, understanding patient experiences, and detecting potential adverse effects in real time. This study advances sentiment analyses for pharmacovigilance by introducing a data-centric framework that incorporates a GenAI-powered labeling system for reliable and interpretable data annotation. A corpus of 213,869 user-generated drug reviews was processed through a hybrid labeling pipeline that reconciles user ratings, lexicon-based polarity, zero-shot transformer predictions, and GPT-5.2 as a fallback mechanism. This strategy enables the resolution of sentiment ambiguity, particularly the frequent misalignment between user-assigned ratings and underlying textual sentiment, by leveraging contextual understanding rather than relying solely on numerical scores. Drug review representations are enhanced using the Qwen3-Embedding-0.6B model, allowing improved capture of semantic nuances. Evaluated through 10-fold stratified cross-validation, the proposed labeling framework combined with a Random Forest classifier achieves a classification accuracy of 96.45%, with per-class analysis confirming consistent performance across all sentiment categories. Cross-source validation on an independent drug review dataset of 4091 reviews and a threshold sensitivity analysis further support the robustness and generalizability of the proposed approach.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Pharmacovigilance and Adverse Drug Reactions · Sentiment Analysis and Opinion Mining · Misinformation and Its Impacts
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Digital Systems, University of Piraeus, 18534 Piraeus, Greece
References (36)
Show source-provided references
1. Artificial Intelligence for Drug Toxicity and Safety10.1016/j.tips.2019.07.005 ↗
2. Towards Drug Safety Surveillance and Pharmacovigilance: Current Progress in Detecting Medication and Adverse Drug Events from Electronic Health Records10.1007/s40264-018-0766-8 ↗
3. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems 30 (NeurIPS 2017), Curran Associates Inc.
4. The Impact of Preprocessing Steps on the Accuracy of Machine Learning Algorithms in Sentiment Analysis10.1007/s10588-018-9266-8 ↗
5. Aspect-Level Drug Reviews Sentiment Analysis Based on Double BiGRU and Knowledge Transfer10.1109/ACCESS.2020.2969473 ↗
6. Vijayaraghavan, S., and Basu, D. (2020). Sentiment Analysis in Drug Reviews using Supervised Machine Learning Algorithms. arXiv.
7. Comparing Deep Learning Architectures for Sentiment Analysis on Drug Reviews10.1016/j.jbi.2020.103539 ↗
8. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, Association for Computational Linguistics.
9. Garg, S. (2021, January 28–29). Drug Recommendation System based on Sentiment Analysis of Drug Reviews using Machine Learning. Proceedings of the 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), Noida, India.10.1109/Confluence51648.2021.9377188 ↗
10. Drug Sentiment Analysis using Machine Learning Classifiers
11. Drug Usage Safety from Drug Reviews with Hybrid Machine Learning Approach10.32604/csse.2023.029059 ↗
12. Zhang, W., Deng, Y., Liu, B., Pan, S.J., and Bing, L. (2023). Sentiment Analysis in the Era of Large Language Models: A Reality Check. arXiv.10.18653/v1/2024.findings-naacl.246 ↗
13. ChatGPT Outperforms Crowd Workers for Text-Annotation Tasks10.1073/pnas.2305016120 ↗
14. Improving Large Language Models for Clinical Named Entity Recognition via Prompt Engineering10.1093/jamia/ocad259 ↗
15. Snorkel: Rapid Training Data Creation with Weak Supervision10.1007/s00778-019-00552-1 ↗
16. Zhang, J., Yu, Y., Li, Y., Wang, Y., Yang, Y., Yang, M., and Ratner, A. (2021). WRENCH: A Comprehensive Benchmark for Weak Supervision. Advances in Neural Information Processing Systems 34 (NeurIPS 2021), Neural Information Processing Systems Foundation, Inc. (NeurIPS).
17. Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective10.1007/s00778-022-00775-9 ↗
18. Sarlis, S., and Maglogiannis, I. (2020). On the Reusability of Sentiment Analysis Datasets in Applications with Dissimilar Contexts. Artificial Intelligence Applications and Innovations (AIAI 2020), Springer.10.1007/978-3-030-49161-1_34 ↗
19. Gräßer, F., Kallumadi, S., Malberg, H., and Zaunseder, S. (2018, January 23–26). Aspect-Based Sentiment Analysis of Drug Reviews Applying Cross-Domain and Cross-Data Learning. Proceedings of the DH 2018, Lyon, France.10.1145/3194658.3194677 ↗
20. (2026, April 15). Drug Review Dataset (Drugs.com). Available online: https://www.kaggle.com/datasets/jessicali9530/kuc-hackathon-winter-2018.
21. Loria, S. (2026, April 15). TextBlob: Simplified Text Processing. Release 0.15, 2018. Available online: https://textblob.readthedocs.io/.
22. Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L. (2020). BART: Denoising Sequence-to-Sequence Pre-Training for Natural Language Generation, Translation, and Comprehension. Proceedings of ACL 2020, Association for Computational Linguistics.10.18653/v1/2020.acl-main.703 ↗
23. Williams, A., Nangia, N., and Bowman, S.R. (2018). A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. Proceedings of NAACL-HLT 2018, Association for Computational Linguistics.10.18653/v1/N18-1101 ↗
24. Enevoldsen, K., Chung, I., Kerboua, I., Kardos, M., Mathur, A., Stap, D., Gala, J., Siblini, W., Krzemiński, D., and Winata, G.I. (2025). MMTEB: Massive Multilingual Text Embedding Benchmark. arXiv.
25. OpenAI (2026, April 15). Update to GPT-5 System Card: GPT-5.2, 2025. Available online: https://openai.com/index/gpt-5-system-card-update-gpt-5-2/.
26. U.S. Food and Drug Administration (2026, April 15). Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan, Available online: https://www.fda.gov/news-events/press-announcements/fda-releases-artificial-intelligencemachine-learning-action-plan.
27. World Health Organization (2026, April 15). Ethics and Governance of Artificial Intelligence for Health, Available online: https://www.who.int/publications/i/item/9789240029200.
28. Random Forests10.1023/A:1010933404324 ↗
29. Chen, T., and Guestrin, C. (2016, January 13–17). XGBoost: A Scalable Tree Boosting System. Proceedings of the KDD 2016, San Francisco, CA, USA.
30. Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.-Y. (2017). LightGBM: A Highly Efficient Gradient Boosting Decision Tree. Advances in Neural Information Processing Systems 30 (NeurIPS 2017), Curran Associates Inc.
31. Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019, January 4–8). Optuna: A Next-Generation Hyperparameter Optimization Framework. Proceedings of the KDD 2019, Anchorage, AK, USA.10.1145/3292500.3330701 ↗
32. Snoek, J., Larochelle, H., and Adams, R.P. (2012). Practical Bayesian Optimization of Machine Learning Algorithms. Advances in Neural Information Processing Systems 25 (NeurIPS 2012), Curran Associates Inc.
33. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., and Xing, E.P. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems 36 (NeurIPS 2023), Curran Associates Inc.10.52202/075280-2020 ↗
34. Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms10.1162/089976698300017197 ↗
35. Althnian, A., AlSaeed, D., Al-Baity, H., Samha, A., Dris, A., Alzakari, N., Abou Elwafa, A., and Kurdi, H. (2021). Impact of Dataset Size on Classification Performance: An Empirical Evaluation in the Medical Domain. Appl. Sci., 11.10.3390/app11020796 ↗
36. (2026, April 15). Drug Review Dataset (DrugLib.com). UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/461/drug+review+dataset+druglib+com.