Leveraging Large Language Models for Departmental Classification of Medical Records
Source abstract
Abstract
This research develops large language models (LLMs) to alleviate the workload of healthcare professionals by classifying medical records into their departments. The models utilize medical records as a dataset for fine-tuning and use clinical knowledge bases to enhance accuracy and efficiency in identifying appropriate departments. This study explores the integration of advanced large language models (LLMs) with quantized low-rank adaptation (QLoRA) for efficient training. The medical department classifier demonstrated impressive performance in diagnosing medical conditions, with an accuracy of 96.26. The findings suggest that LLM-based solutions could significantly improve the efficiency of clinical consultations. What is more, the trained models are hosted on GitHub and are publicly available for use, empowering the wider community to benefit from this research.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Artificial Intelligence in Healthcare and Education · AI in cancer detection · Topic Modeling
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
IANLP, Computer Science, Wenzhou-Kean University, Daxue 88, Wenzhou 325060, China
Department of Computer Science and Technology, Kean University, New Jersey, NJ 07083, USA
Funding
Wenzhou-Kean University
References (33)
Show source-provided references
1. Sosialisasi Pentingnya Rekam Medis Di Fasilitas Layanan Kesehatan Pada Siswa Sman 1 Teluk Sebong
2. Liu, F., Li, Z., Zhou, H., Yin, Q., Yang, J., Tang, X., Luo, C., Zeng, M., Jiang, H., and Gao, Y. (2024). Large Language Models in the Clinic: A Comprehensive Benchmark. arXiv.
3. Yang, J., Jin, H., Tang, R., Han, X., Feng, Q., Jiang, H., Zhong, S., Yin, B., and Hu, X. (2023). Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond. arXiv.10.1145/3649506 ↗
4. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020). Language Models are Few-Shot Learners. arXiv.
5. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., and Azhar, F. (2023). LLaMA: Open and Efficient Foundation Language Models. arXiv.
6. Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Zhang, D., Rojas, D., Feng, G., Zhao, H., and Lai, H. (2024). ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. arXiv.
7. A large language model for electronic health records10.1038/s41746-022-00742-2 ↗
8. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv.
9. QLORA: Efficient Finetuning of Quantized LLMs
10. A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges10.1109/ACCESS.2024.3365742 ↗
11. Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., and Huang, F. (2023). Qwen Technical Report. arXiv.
12. Sequential Labeling with Structural SVM Under Nondecomposable Losses10.1109/TNNLS.2017.2757504 ↗
13. Demirkaya, A., Chen, J., and Oymak, S. (2020, January 18–20). Exploring the Role of Loss Functions in Multiclass Classification. Proceedings of the 2020 54th Annual Conference on Information Sciences and Systems (CISS), Princeton, NJ, USA.10.1109/CISS48834.2020.1570627167 ↗
14. Evaluating large language models on medical evidence summarization10.1038/s41746-023-00896-7 ↗
15. Li, S.S., Balachandran, V., Feng, S., Ilgen, J., Pierson, E., Koh, P.W., and Tsvetkov, Y. (2024). MEDIQ: Question-Asking LLMs for Adaptive and Reliable Clinical Reasoning. arXiv.
16. Huang, K., Altosaar, J., and Ranganath, R. (2020). ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission. arXiv.
17. ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge
18. Ye, Q., Liu, J., Chong, D., Zhou, P., Hua, Y., Liu, F., Cao, M., Wang, Z., Cheng, X., and Lei, Z. (2024). Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model. arXiv.
19. FreedomIntelligence (2024, November 10). Huatuo26M-Lite. [Dataset]. Available online: https://huggingface.co/datasets/FreedomIntelligence/Huatuo26M-Lite.
20. Zeng, X. (2024, November 12). Zeng981/Nlpdataset. [Dataset]. Available online: https://huggingface.co/datasets/zeng981/nlpdataset.
21. Zaheer, R., Shaziya, H., and College, N. (2019, January 10–11). A Study of the Optimization Algorithms in Deep Learning. Proceedings of the 2019 Third International Conference on Inventive Systems and Control (ICISC), Coimbatore, India.10.1109/ICISC44355.2019.9036442 ↗
22. Gemma Team, Riviere, M., Pathak, S., Sessa, P.G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., and Ramé, A. (2024). Gemma 2: Improving Open Language Models at a Practical Size. arXiv.
23. Ainslie, J., Lee-Thorp, J., Jong, M., Zemlyanskiy, Y., Lebrón, F., and Sanghai, S. (2023). GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. arXiv.10.18653/v1/2023.emnlp-main.298 ↗
24. Beltagy, I., Peters, M.E., and Cohan, A. (2020). Longformer: The Long-Document Transformer. arXiv.
25. Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., and Huang, F. (2024). Qwen2 Technical Report. arXiv.
26. Lin, C.-Y. (2025, May 27). ROUGE: A Package for Automatic Evaluation of Summaries. Available online: https://aclanthology.org/W04-1013/.
27. Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics—ACL ’02. Association for Computational Linguistics, Philadelphia, PA, USA, 6–12 July 2002.10.3115/1073083.1073135 ↗
28. Psycholinguistic Norms for 3,783 Two-Character Words in Simplified Chinese10.1177/21582440211054495 ↗
29. Topol, E. (2019). Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again, Hachette.
30. Artificial intelligence in healthcare10.1038/s41551-018-0305-z ↗
31. The future of digital health with federated learning10.1038/s41746-020-00323-1 ↗
32. A Survey on Bias and Fairness in Machine Learning10.1145/3457607 ↗
33. Towards continual learning: A survey