Fine-Tuning a Personalized OpenBioLLM Using Offline Reinforcement Learning

In the past decade, personalized treatment has become increasingly significant; however, the individualized care provided by physicians incurs substantial labor costs. The advancement of artificial intelligence, particularly the emergence of large language models, offers promising prospects for personalized treatment. These models have the potential to create intelligent agents capable of delivering tailored medical care. Nonetheless, current large language models face challenges in achieving precise treatment recommendations and optimizing treatment strategies. This study aims to enhance the accuracy of medical large language models by employing representation learning, inverse reinforcement learning, and offline reinforcement learning techniques for fine-tuning. The fine-tuned models are capable of generating personalized treatment plans. Through experiments on the mechanical ventilation therapy dataset constructed from the MIMIC-III database, we found that the similarity between the treatment actions suggested by the fine-tuned models and those of clinical physicians reaches 90.4% in the survival dataset, while it decreases to 72.3% in the mortality dataset. These results suggest that fine-tuned models are able to effectively capture individual patient information and provide treatment strategies that differ from routine clinical practice in complex situations, rather than merely mimicking physician behavior. Nevertheless, we emphasize that the recommendations of the model should be used as an auxiliary tool for physician decision making, and the final medical decision still needs to be comprehensively considered in combination with the patient’s clinical background, medical history, and autonomy.