Accuracy of the Sentence-BERT Semantic Search System for a Japanese Database of Closed Medical Malpractice Claims
Source abstract
Abstract
In this study, we developed a similar text retrieval system using Sentence-BERT (SBERT) for our database of closed medical malpractice claims and investigated its retrieval accuracy. We assigned each case in the database a short Japanese summary of the accident as well as two labels: the category was classified as a hospital department mainly, and the process indicated a failed medical procedure. We evaluated the accuracies of a similar text retrieval system with the two labels using three different multilabel evaluation metrics. For the encoders of SBERT, we employed two pretrained BERT models, UTH-BERT and NICT-BERT, that were trained on huge Japanese corpora, and we performed iterative optimization to train the SBERTs. The accuracies of the similar text retrieval systems using the trained SBERTs were more than 15 points higher than those of the Okapi BM25 system and the pretrained SBERT system.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Artificial Intelligence in Law · Biomedical Text Mining and Ontologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
General Medical Education and Research Center, Teikyo University, Tokyo 173-8605, Japan
Funding
JSPS KAKENHI
References (25)
Show source-provided references
1. (2023, January 31). The Project to Collect Medical Near-Miss/Adverse Event Information. Available online: https://www.med-safe.jp/.
2. Japan Medical Safety Research Organization (2023, January 31). To Secure Medical Safety “Medical Accident Investigation System” in Japan. Available online: https://www.medsafe.or.jp/uploads/uploads/files/leaflet_english.pdf.
3. (2023, January 31). Medical Professional Liability Association. Available online: https://www.mplassociation.org/.
4. Analysis of closed claims in the clinical management of rheumatoid arthritis in Japan10.4103/0366-6999.207479 ↗
5. Characteristics of claims in the management of septic arthritis in Japan: Retrospective analyses of judicial precedents and closed claims10.1016/j.jcma.2017.06.023 ↗
6. Otaki, Y., Fujishiro, N., Oyama, Y., Hata, N., Kato, D., and Kawachi, S. (2022). The risk of fatal bleeding complications in jugular catheterization in patients with coagulopathy: A retrospective analysis of death cases in closed claims and the Medical Accident Investigating System in Japan. PLoS ONE, 17.10.1371/journal.pone.0261636 ↗
7. Robertson, S.E., and Walker, S. (1994, January 3–6). Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. Proceedings of the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, Dublin, Ireland.10.1007/978-1-4471-2099-5_24 ↗
8. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013, January 2–4). Efficient estimation of word representations in vector space. Proceedings of the Workshop at the International Conference on Learning Representations, Scottsdale, AZ, USA.
9. Le, Q., and Mikolov, T. (2014, January 21–26). Distributed representations of sentences and documents. Proceedings of the 31st International Conference on Machine Learning (ICML 2014), Beijing, China.
10. Pennington, J., Socher, R., and Manning, C.D. (2014, January 25–29). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), Doha, Qatar.10.3115/v1/D14-1162 ↗
11. Bojanowski, P., Grave, E., Joulin, A., and Mikolov, T. (2016). Enriching word vectors with subword information. arXiv.10.1162/tacl_a_00051 ↗
12. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2–7). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.
13. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020, January 6–12). Language models are few-shot learners. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
14. Reimers, N., and Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. arXiv.10.18653/v1/D19-1410 ↗
15. (2023, February 15). Sentence-Transformers. Available online: https://www.sbert.net/.
16. Kawazoe, Y., Shibata, D., Shinohara, E., Aramaki, E., and Ohe, K. (2021). A clinical specific BERT developed using a huge Japanese clinical text corpus. PLoS ONE, 16.10.1371/journal.pone.0259763 ↗
17. (2023, January 31). NICT BERT: A Japanese Pre-Trained Model. Available online: https://alaginrc.nict.go.jp/nict-bert/index.html.
18. (2023, January 31). MeCab: Yet Another Part-of-Speech and Morphological Analyzer. Available online: https://taku910.github.io/mecab/.
19. SlothLib: A programming library for research on web search
20. Improving document clustering using Okapi BM25 feature weighting10.1007/s10791-011-9163-y ↗
21. Sorower, M.S. (2010). A Literature Survey on Algorithms for Multi-Label Learning. [Ph.D. Thesis, Computer Science, Oregon State University].
22. Henderson, M., Al-Rfou, R., Strope, B., Sung, Y.H., Lukács, L., Guo, R., Kumar, S., Miklos, B., and Kurzweil, R. (2017). Efficient natural language response suggestion for smart reply. arXiv.
23. Hossain, T., Logan IV, R.L., Ugarte, A., Matsubara, Y., Young, S., and Singh, S. (2020, January 16–20). COVIDLIES: Detecting COVID-19 misinformation on social media. Proceedings of the Workshop on NLP for COVID-19 (Part 2), EMNLP, Virtual.10.18653/v1/2020.nlpcovid19-2.11 ↗
24. Mass, Y., Carmeli, B., Roitman, H., and Konopnicki, D. (2020, January 5–10). Unsupervised FAQ retrieval with question generation and BERT. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Virtual.10.18653/v1/2020.acl-main.74 ↗
25. Shibata, D., Kawazoe, Y., Shimamoto, K., Shinohara, E., and Aramaki, E. (2020, January 16–19). Pain expressions extraction with pre-training BERT based on clinical note. Proceedings of the Spring Meeting on Medical Informatics, Mito, Japan.10.1371/journal.pone.0259763 ↗