A Neural N-Gram-Based Classifier for Chinese Clinical Named Entity Recognition
Source abstract
Abstract
Clinical Named Entity Recognition (CNER) focuses on locating named entities in electronic medical records (EMRs) and the obtained results play an important role in the development of intelligent biomedical systems. In addition to the research in alphabetic languages, the study of non-alphabetic languages has attracted considerable attention as well. In this paper, a neural model is proposed to address the extraction of entities from EMRs written in Chinese. To avoid erroneous noise being caused by the Chinese word segmentation, we employ the character embeddings as the only feature without extra resources. In our model, concatenated n-gram character embeddings are used to represent the context semantics. The self-attention mechanism is then applied to model long-range dependencies of embeddings. The concatenation of the new representations obtained by the attention module is taken as the input to bidirectional long short-term memory (BiLSTM), followed by a conditional random field (CRF) layer to extract entities. The empirical study is conducted on the CCKS-2017 Shared Task 2 dataset to evaluate our method and the experimental results show that our model outperforms other approaches.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Biomedical Text Mining and Ontologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Master Program of Digital Innovation, Tunghai University, Taichung 40704, Taiwan
Department of Computer Science, Tunghai University, Taichung 40704, Taiwan
References (36)
Show source-provided references
1. Hierarchical hidden markov models for information extraction
2. EliIE: An open-source information extraction system for clinical trial eligibility criteria10.1093/jamia/ocx019 ↗
3. Yadav, V., and Bethard, S. (2018, January 21–25). A Survey on Recent Advances in Named Entity Recognition from Deep Learning models. Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, NM, USA.
4. A comparative study for biomedical named entity recognition10.1007/s13042-015-0426-6 ↗
5. Hu, J., Shi, X., Liu, Z., Wang, X., Chen, Q., and Tang, B. (2017). HITSZ_CNER: A Hybrid System for Entity Recognition from Chinese Clinical Text, CEUR Workshop Proceedings.
6. Li, L., Zhao, J., Hou, L., Zhai, Y., Shi, J., and Cui, F. (2019). An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records. BMC Med. Inform. Decis. Mak., 19.10.1186/s12911-019-0933-6 ↗
7. Clinical Named Entity Recognition from Chinese Electronic Medical Records Based on Deep Learning Pretraining10.1155/2020/8829219 ↗
8. An Attention-Based BiLSTM-CRF Model for Chinese Clinic Named Entity Recognition10.1109/ACCESS.2019.2935223 ↗
9. GRAM-CNN: A deep learning approach with local context for named entity recognition in biomedical text10.1093/bioinformatics/btx815 ↗
10. Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition10.1016/j.jbi.2019.103133 ↗
11. MAF-CNER: A Chinese Named Entity Recognition Model Based on Multifeature Adaptive Fusion10.1155/2021/6696064 ↗
12. Zeng, Q.T., Goryachev, S., Weiss, S., Sordo, M., Murphy, S.N., and Lazarus, R. (2006). Extracting principal diagnosis, co-morbidity and smoking status for asthma research: Evaluation of a natural language processing system. BMC Med. Inform. Decis. Mak., 6.10.1186/1472-6947-6-30 ↗
13. Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): Architecture, component evaluation and applications10.1136/jamia.2009.001560 ↗
14. EDGAR: Extraction of drugs, genes and relations from the biomedical literature
15. Aronson, A.R. (2001, January 3–7). Effective Mapping of Biomedical Text to the UMLS Metathesaurus: The MetaMap Program. Proceedings of the AMIA Symposium, Washington, DC, USA.
16. Gaizauskas, R., Demetriou, G., and Humphreys, K. (2000, January 2–4). Term recognition and classification in biological science journal articles. Proceedings of the Computional Terminology for Medical and Biological Applications Workshop of the 2nd International Conference on NLP, Patras, Greece.
17. The Regenstrief medical record system: A quarter century experience10.1016/S1386-5056(99)00009-X ↗
18. SNOMED-CT: The advanced terminology and coding system for eHealth
19. Supervised methods for symptom name recognition in free-text clinical records of traditional Chinese medicine: An empirical study10.1016/j.jbi.2013.09.008 ↗
20. Ju, Z., Wang, J., and Zhu, F. (2011, January 10–12). Named entity recognition from biomedical text using SVM. Proceedings of the 2011 5th International Conference on Bioinformatics and Biomedical Engineering, Wuhan, China.10.1109/icbbe.2011.5779984 ↗
21. Yin, W., Kann, K., Yu, M., and Schütze, H. (2017). Comparative study of CNN and RNN for natural language processing. arXiv.
22. Li, Z., Zhang, Q., Liu, Y., Feng, D., and Huang, Z. (2017). Recurrent Neural Networks with Specialized Word Embedding for Chinese Clinical Named Entity Recognition, CEUR Workshop Proceedings.
23. Ouyang, E., Li, Y., Jin, L., Li, Z., and Zhang, X. (2017). Exploring N-Gram Character Presentation in Bidirectional RNN-CRF for Chinese Clinical Named Entity Recognition, CEUR Workshop Proceedings.
24. Attention is All you Need
25. Tan, Z., Wang, M., Xie, J., Chen, Y., and Shi, X. (2018, January 2–7). Deep semantic role labeling with self-attention. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11928 ↗
26. Deformable Self-Attention for Text Classification10.1109/TASLP.2021.3067210 ↗
27. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
28. Lafferty, J., McCallum, A., and Pereira, F.C. (July, January 28). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of the 18th International Conference on Machine Learning, Williamstown, MA, USA.
29. Alzaidy, R., Caragea, C., and Giles, C.L. (2019, January 13–17). Bi-LSTM-CRF sequence labeling for keyphrase extraction from scholarly documents. Proceedings of the World Wide Web Conference, San Francisco, CA, USA.10.1145/3308558.3313642 ↗
30. Ma, X., and Hovy, E. (2016). End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv.10.18653/v1/P16-1101 ↗
31. Chinese clinical named entity recognition with variant neural structures based on BERT methods10.1016/j.jbi.2020.103422 ↗
32. Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition10.1016/j.jbi.2017.11.007 ↗
33. Qiu, J., Wang, Q., Zhou, Y., Ruan, T., and Gao, J. (2018, January 3–6). Fast and accurate recognition of Chinese clinical named entities with residual dilated convolutions. Proceedings of the 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Madrid, Spain.10.1109/BIBM.2018.8621360 ↗
34. Strubell, E., Verga, P., Belanger, D., and McCallum, A. (2017). Fast and accurate entity recognition with iterated dilated convolutions. arXiv.10.18653/v1/D17-1283 ↗
35. Adversarial training based lattice LSTM for Chinese clinical named entity recognition10.1016/j.jbi.2019.103290 ↗
36. Jiang, S., Zhao, S., Hou, K., Liu, Y., and Zhang, L. (2019, January 26–27). A bert-bilstm-crf model for chinese electronic medical records named entity recognition. Proceedings of the 2019 12th International Conference on Intelligent Computation Technology and Automation (ICICTA), Xiangtan, China.