Deep Reinforcement Learning-Driven Adaptive Prompting for Robust Medical LLM Evaluation
Source abstract
Abstract
The accurate and reliable evaluation of large language models (LLMs) in medical domains is critical for real-world clinical deployment, automated medical reasoning, and patient safety. However, the evaluation process is highly sensitive to prompt design, and prevalent reliance on fixed or randomly sampled prompt policies often fails to dynamically adapt to clinical context, question complexity, or evolving safety requirements. This article presents a novel reinforcement learning-based framework for multi-prompt selection, which dynamically optimizes prompt policy per input for medical LLM evaluation across the Medical Knowledge Question-Answering dataset (MKQA), the Medical Multiple-Choice Question dataset (MCQ), and the Doctor-Patient Dialogue dataset. We formulate prompt selection as a Markov Decision Process (MDP) and employ a deep Q-Network (DQN) agent to maximize a reward signal incorporating textual accuracy, domain terminology coverage, safety, and dialogue relevance. Experiments on three medical LLM benchmarks demonstrate consistent improvements in composite reward (e.g., a 6.66% increase in MKQA vs. Random Baseline, and a 2.41% increase in Dialogue vs. Fixed Baseline) when compared to baselines. This was accompanied by robust enhancements in Safety (e.g., achieving 1.0000 in MKQA, a 5.26% increase vs. Fixed Baseline; and a 5.03% increase in Dialogue vs. Fixed Baseline) and substantial gains in Medical Terminology Coverage (e.g., a 74.61% increase in MKQA vs. Fixed Baseline, and a 9.13% increase in MCQ vs. Fixed Baseline) when compared to baselines. While varying across tasks, an improvement in accuracy was observed in the MKQA task, and the framework effectively optimizes the multi-objective reward function, even when minor trade-offs in other metrics like Accuracy and Contextual Relevance were observed in some contexts. Our framework enables robust, context-aware, and adaptive evaluation, laying a foundation for safer and more reliable LLM application in healthcare.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Healthcare · Topic Modeling · Artificial Intelligence in Healthcare and Education
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Data Science, The Chinese University of Hong Kong, Shenzhen, Shenzhen 518172, China
Intelligence Institute of Machine, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China
iFLYTEK Co., Ltd., Hefei 230088, China
Funding
National Key Research and Development Program of China · the Anhui Provincial Major Science and Technology Project · the Anhui Provincial Clinical Medical Research Transformation Project · the Research Funds of Center for Xin'an Medicine and Modernization of Traditional Chinese Medicine of IHM
References (45)
Show source-provided references
1. Detecting hallucinations in large language models using semantic entropy10.1038/s41586-024-07421-0 ↗
2. A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation10.1038/s41746-025-01670-7 ↗
3. The application of large language models in medicine: A scoping review10.1016/j.isci.2024.109713 ↗
4. Cardiovascular safety in oncology clinical trials: JACC: CardioOncology Primer
5. The cancer patient and cardiology10.1002/ejhf.1985 ↗
6. Prompt engineering paradigms for medical applications: Scoping review10.2196/60501 ↗
7. An empirical evaluation of prompting strategies for large language models in zero-shot clinical natural language processing: Algorithm development and validation study10.2196/55318 ↗
8. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing Atari with deep reinforcement learning. arXiv.
9. Toward expert-level medical question answering with large language models10.1038/s41591-024-03423-7 ↗
10. A generalist medical language model for disease diagnosis assistance10.1038/s41591-024-03416-6 ↗
11. Michalopoulos, G., Williams, K., Singh, G., and Lin, T. (2022, January 7–11). MedicalSum: A guided clinical abstractive summarization model for generating medical reports from patient-doctor conversations. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates.10.18653/v1/2022.findings-emnlp.349 ↗
12. An artificial intelligence-based chatbot for prostate cancer education: Design and patient evaluation study10.1177/20552076231173304 ↗
13. Attribute-based classification for zero-shot visual object categorization10.1109/TPAMI.2013.140 ↗
14. Language models are few-shot learners
15. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
16. Wei, J., Bosma, M., Zhao, V.Y., Guu, K., Yu, A.W., Lester, B., Du, N., Dai, A.M., and Le, Q.V. (2021). Finetuned language models are zero-shot learners. arXiv.
17. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing
18. Christiano, P.F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. (2017). Deep Reinforcement Learning from Human Preferences. Proceedings of the Advances in Neural Information Processing Systems, Curran Associates, Inc.
19. Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., and McKinnon, C. (2022). Constitutional AI: Harmlessness from AI feedback. arXiv.
20. Wang, X., Li, C., Wang, Z., Bai, F., Luo, H., Zhang, J., Jojic, N., Xing, E.P., and Hu, Z. (2023). PromptAgent: Strategic planning with language models enables expert-level prompt optimization. arXiv.
21. Wang, X., Chen, N., Chen, J., Hu, Y., Wang, Y., Wu, X., Gao, A., Wan, X., Li, H., and Wang, B. (2024). Apollo: Lightweight Multilingual Medical LLMs towards Democratizing Medical AI to 6B People. arXiv.
22. Yang, S., Zhao, H., Zhu, S., Zhou, G., Xu, H., Jia, Y., and Zan, H. (2023). Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue. arXiv.10.1609/aaai.v38i17.29907 ↗
23. Chen, Z. (2025, December 02). QASystemOnMedicalGraph: A Medical Knowledge Graph Based Question Answering System. Available online: https://github.com/zhihao-chen/QASystemOnMedicalGraph.
24. Xtea (2025, December 02). Chinese Medical Words. Available online: https://github.com/xtea/chinese_medical_words.
25. (2025, December 02). 33211. Chinese Stopwords List. Available online: https://github.com/33211/stopwords.
26. DeepSeek-AI (2024). DeepSeek-V3 Technical Report. arXiv.
27. OpenAI (2025, December 02). Introducing GPT-4.1 in the API. Available online: https://openai.com/index/gpt-4-1/.
28. Buitrago, P.A., and Nystrom, N.A. (August, January 28). Open compass: Accelerating the adoption of AI in open research. Proceedings of the Practice and Experience in Advanced Research Computing 2019: Rise of the Machines (Learning), Chicago, IL, USA.10.1145/3332186.3332253 ↗
29. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models
30. Zhang, N., Chen, M., Bi, Z., Liang, X., Li, L., Shang, X., Yin, K., Tan, C., Xu, J., and Huang, F. (2021). Cblue: A chinese biomedical language understanding evaluation benchmark. arXiv.
31. Xu, L., Li, A., Zhu, L., Xue, H., Zhu, C., Zhao, K., He, H., Zhang, X., Kang, Q., and Lan, Z. (2023). Superclue: A comprehensive chinese large language model benchmark. arXiv.
32. Liao, Y., Jiang, S., Wang, Y., and Wang, Y. (2024). MING-MOE: Enhancing Medical Multi-Task Learning in Large Language Models with Sparse Mixture of Low-Rank Adapter Experts. arXiv.
33. Chen, J., Cai, Z., Ji, K., Wang, X., Liu, W., Wang, R., Hou, J., and Wang, B. (2024). HuatuoGPT-o1, towards medical complex reasoning with LLMs. arXiv.
34. Taiyi: A bilingual fine-tuned large language model for diverse biomedical tasks10.1093/jamia/ocae037 ↗
35. Zhang, X., Xue, K., and Zhang, S. (2025, December 02). PULSE: Pretrained and Unified Language Service Engine. Available online: https://github.com/openmedlab/PULSE.
36. Hu, S., Tu, Y., Han, X., He, C., Cui, G., Long, X., Zheng, Z., Fang, Y., Huang, Y., and Zhao, W. (2024). MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies. arXiv.
37. Young, A., Chen, B., Li, C., Huang, C., Zhang, G., Zhang, G., Wang, G., Li, H., Zhu, J., and Chen, J. (2024). Yi: Open foundation models by 01. ai. arXiv.
38. DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv.
39. Cui, Y., Yang, Z., and Yao, X. (2023). Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca. arXiv.
40. Ribeiro, M.T., Singh, S., and Guestrin, C. (2016). Model-agnostic interpretability of machine learning. arXiv.
41. Zhang, K., Yang, Z., and Başar, T. (2021). Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, Springer.10.1007/978-3-030-60990-0_12 ↗
42. Goyal, A., Friesen, A., Banino, A., Weber, T., Ke, N.R., Badia, A.P., Guez, A., Mirza, M., Humphreys, P.C., and Konyushova, K. (2022, January 17–23). Retrieval-augmented reinforcement learning. Proceedings of the International Conference on Machine Learning, Baltimore, MD, USA.
43. Fernando, C., Banarse, D., Michalewski, H., Osindero, S., and Rocktäschel, T. (2023). Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv.
44. Li, L., Zhang, Y., and Chen, L. (2023, January 21–25). Prompt distillation for efficient LLM-based recommendation. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, Birmingham, UK.10.1145/3583780.3615017 ↗
45. Zhang, Y., Yuan, Y., and Yao, A.C. (2023). Meta prompting for AI systems. arXiv.