Research on Medical Text Parsing Method Based on BiGRU-BiLSTM Multi-Task Learning
Source abstract
Abstract
With the development of technology, the popularity of online medical treatment is becoming more and more widespread. However, the accuracy and credibility of online medical treatment are affected by model design and semantic understanding. In particular, there are still some problems in the accurate understanding of complex structured texts, which affects the accuracy of judging users’ intentions and needs. Therefore, this paper proposes a new method for medical text parsing, which realizes core tasks such as named entity recognition, intention recognition, and slot filling through a multi-task learning framework; uses BERT to obtain contextual semantic information; and combines BiGRU and BiLSTM neural networks, and uses CRF to realize sequence annotation and DPCNN to realize classification prediction. Thus, the task of entity recognition and intention recognition can be accomplished. On this basis, this paper builds a multi-task learning model based on BiGRU-BiLSTM, and uses CBLUE and CMID databases to verify the method. The verification results show that the accuracy of named entity recognition and intention recognition reaches 86% and 89%, respectively, which improves the performance of various tasks. The ability of the model to process complex text is enhanced. This method can improve the text generalization ability and improve the accuracy of online medical intelligent dialogue when it is used to analyze medical texts.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques · Topic Modeling · Web Data Mining and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China
References (31)
Show source-provided references
1. He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., and Cambria, E. (2023). A survey of large language models for healthcare: From data, technology, and applications to accountability and ethics. arXiv.10.2139/ssrn.4809363 ↗
2. Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., and Neal, D. (2023). Towards expert-level medical question answering with large language models. arXiv.
3. Ontology-based enriched concept graphs for medical document classification10.1016/j.ins.2020.03.006 ↗
4. Lenivtceva, I., Slasten, E., Kashina, M., and Kopanitsa, G. (2020, January 3–5). Applicability of machine learning methods to multi-label medical text classification. Proceedings of the Computational Science–ICCS 2020: 20th International Conference, Amsterdam, The Netherlands. Proceedings, Part IV.10.1007/978-3-030-50423-6_38 ↗
5. Chinese medical question answer selection via hybrid models based on CNN and GRU10.1007/s11042-019-7240-1 ↗
6. Shi, H., Liu, X., Shi, G., Li, D., and Ding, S. (2023, January 20–22). Research on medical automatic Question answering model based on knowledge graph. Proceedings of the 2023 35th Chinese Control and Decision Conference (CCDC), Yichang, China.10.1109/CCDC58219.2023.10327124 ↗
7. Meiling, W., Xiaohai, H., Yan, L., Linbo, Q., Zhao, Z., and Honggang, C. (2023). MAGE: Multi-scale Context-aware Interaction based on Multi-granularity Embedding for Chinese Medical Question Answer Matching. Comput. Methods Programs Biomed., 228.10.1016/j.cmpb.2022.107249 ↗
8. Huang, X. (2022). Design and Implementation of Medical Question Answering System Based on ALBERT. [Master’s Thesis, University of Electronic Science and Technology of China].
9. ESIE-BERT: Enriching sub-words information explicitly with BERT for intent classification and slot filling10.1016/j.neucom.2024.127725 ↗
10. The joint model of multi-intention recognition and slot filling of GL-GIN is improved
11. Intelligent medical question answering system based on BERT-BiGRU model
12. Chinese medical question answering system based on BERT
13. Hu, W., and Zhang, Y. (2023). Medical entity recognition method based on BERT-BiGRU-CRF. Comput. Age, 24–27.
14. Kollias, D. (2022, January 23–27). ABAW: Learning from Synthetic Data & Multi-task Learning Challenges. Proceedings of the Computer Vision—ECCV 2022 Workshops, Tel Aviv, Israel.10.1007/978-3-031-25075-0_12 ↗
15. Vandenhende, S., Georgoulis, S., Proesmans, M., Dai, D., and Van Gool, L. (2020). Revisiting multi-task learning in the deep learning era. arXiv.
16. MTL-BERT: A Multi-task Learning Model Utilizing Bert for Chinese Text
17. Zhang, J. (2022). Research on Named Entity Recognition Method for Chinese Electronic Medical Records. [Master’s Thesis, Zhejiang University of Science and Technology].
18. Research on Named Named entity recognition Based on Multi-Task Learning and Biaffine Mechanism10.1155/2022/2687615 ↗
19. CopyMTL: Copy Mechanism for Joint Extraction of Entities and Relations with Multi-Task Learning
20. Peng, Y., Chen, Q., and Lu, Z. (2020). An empirical study of multi-task learning on BERT for biomedical text mining. arXiv.10.18653/v1/2020.bionlp-1.22 ↗
21. Chinese named entity recognition based on tag information fusion and multi-task learning
22. Li, Y. (2023). Speech Emotion Recognition with Multitask Learning. [Master’s Thesis, China University of Mining and Technology].
23. Multi-task learning for aspect level semantic classification combining complex aspect target semantic enhancement and adaptive local focus10.3934/mbe.2023824 ↗
24. Unveiling the dynamics of crisis events: Sentiment and emotion analysis via multi-task learning with attention mechanism and subject-based intent prediction10.1016/j.ipm.2024.103695 ↗
25. Song, D., Hu, M., Ding, J., Qu, Z., Chang, Z., and Qian, L. (2024). Research on cross-type text classification technology based on multi-task learning. Data Anal. Knowl. Discov., 1–19.
26. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv.
27. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
28. Cho, K. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.10.3115/v1/D14-1179 ↗
29. Johnson, R., and Zhang, T. (August, January 30). Deep pyramid convolutional neural networks for text categorization. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BC, Canada.
30. Zhang, N., Chen, M., Bi, Z., Liang, X., Li, L., Shang, X., Yin, K., Tan, C., Xu, J., and Huang, F. (2021). Cblue: A chinese biomedical language understanding evaluation benchmark. arXiv.10.18653/v1/2022.acl-long.544 ↗
31. Chen, N., Su, X., Liu, T., Hao, Q., and Wei, M. (2020). A benchmark dataset and case study for Chinese medical question intent classification. BMC Med. Inform. Decis. Mak., 20.10.1186/s12911-020-1122-3 ↗