Comparing Deep Learning and Classical Machine Learning Approaches for Predicting Inpatient Violence Incidents from Clinical Text
Source abstract
Abstract
Machine learning techniques are increasingly being applied to clinical text that is already captured in the Electronic Health Record for the sake of delivering quality care. Applications for example include predicting patient outcomes, assessing risks, or performing diagnosis. In the past, good results have been obtained using classical techniques, such as bag-of-words features, in combination with statistical models. Recently however Deep Learning techniques, such as Word Embeddings and Recurrent Neural Networks, have shown to possibly have even greater potential. In this work, we apply several Deep Learning and classical machine learning techniques to the task of predicting violence incidents during psychiatric admission using clinical text that is already registered at the start of admission. For this purpose, we use a novel and previously unexplored dataset from the Psychiatry Department of the University Medical Center Utrecht in The Netherlands. Results show that predicting violence incidents with state-of-the-art performance is possible, and that using Deep Learning techniques provides a relatively small but consistent improvement in performance. We finally discuss the potential implication of our findings for the psychiatric practice.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Healthcare · Digital Mental Health Interventions · Electronic Health Records Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information and Computing Sciences, Utrecht University, P.O. Box 80089, 3508 TB Utrecht, The Netherlands
Department of Psychiatry, University Medical Center Utrecht, P.O. Box 85500, 3508 GA Utrecht, The Netherlands
References (72)
Show source-provided references
1. EHR Adoption and Hospital Performance: Time-Related Effects10.1111/1475-6773.12406 ↗
2. Transformational Impact of Health Information Technology on the Clinical Practice of Child and Adolescent Psychiatry10.1016/j.chc.2016.07.003 ↗
3. Transitioning to a Data Driven Mental Health Practice: Collaborative Expert Sessions for Knowledge and Hypothesis Finding10.1155/2016/9089321 ↗
4. A survey on big data analytics in health care
5. Big data in health care: Using analytics to identify and manage high-risk and high-cost patients10.1377/hlthaff.2014.0041 ↗
6. Medical big data: Promise and challenges10.23876/j.krcp.2017.36.1.3 ↗
7. The Inevitable Application of Big Data to Health Care10.1001/jama.2013.393 ↗
8. Gaming the quantified self10.24908/ss.v11i1/2.4454 ↗
9. Overcoming barriers to NLP for clinical text: The role of shared tasks and the need for additional creative solutions10.1136/amiajnl-2011-000465 ↗
10. Ford, E., Stockdale, J., Jackson, R., and Cassell, J. (2017). For the greater good? Patient and public attitudes to use of medical free text data in research. Int. J. Popul. Data Sci., 1.10.23889/ijpds.v1i1.249 ↗
11. Genetic validation of bipolar disorder identified by automated phenotyping using electronic health records10.1038/s41398-018-0133-7 ↗
12. Semi-supervised clinical text classification with Laplacian SVMs: An application to cancer case management10.1016/j.jbi.2013.06.014 ↗
13. Pestian, J.P., Brew, C., Matykiewicz, P., Hovermale, D.J., Johnson, N., Cohen, K.B., and Duch, W. (2007, January 29). A shared task involving multi-label classification of clinical free text. Proceedings of the Workshop on BioNLP 2007: Biological, Translational, and Clinical Language Processing, Prague, Czech Republic.10.3115/1572392.1572411 ↗
14. Portable automatic text classification for adverse drug reaction detection via multi-corpus training10.1016/j.jbi.2014.11.002 ↗
15. Mikolov, T., Corrado, G., Chen, K., and Dean, J. (arXiv, 2013). Efficient Estimation of Word Representations in Vector Space, arXiv.
16. Le, Q., and Mikolov, T. (arXiv, 2014). Distributed representations of sentences and documents, arXiv.
17. A primer on neural network models for natural language processing10.1613/jair.4992 ↗
18. Deep learning10.1038/nature14539 ↗
19. Suresh, H., Hunt, N., Johnson, A., Celi, L.A., Szolovits, P., and Ghassemi, M. (arXiv, 2017). Clinical Intervention Prediction and Understanding using Deep Networks, arXiv.
20. Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records10.1038/srep26094 ↗
21. Clinical Relation Extraction with Deep Learning
22. Named Entity Recognition in Chinese Clinical Text Using Deep Neural Network
23. Ekbal, A., Saha, S., and Bhattacharyya, P. (2016, January 11–17). Deep Learning Architecture for Patient Data De-identification in Clinical Records. Proceedings of the Clinical Natural Language Processing Workshop, Osaka, Japan.
24. Shickel, B., Tighe, P.J., Bihorac, A., and Rashidi, P. (2017). Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis. IEEE J. Biomed. Health Inform., 1.10.1109/JBHI.2017.2767063 ↗
25. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs10.1001/jama.2016.17216 ↗
26. Dermatologist-level classification of skin cancer with deep neural networks10.1038/nature21056 ↗
27. Lipton, Z.C., Kale, D.C., Elkan, C., and Wetzel, R. (arXiv, 2015). Learning to Diagnose with LSTM Recurrent Neural Networks, arXiv.
28. Hammerla, N.Y., Halloran, S., and Ploetz, T. (arXiv, 2016). Deep, Convolutional, and Recurrent Models for Human Activity Recognition using Wearables, arXiv.
29. Jacobson, O., and Dalianis, H. (2016, January 12). Applying deep learning on electronic health records in Swedish to predict healthcare-associated infections. Proceedings of the 15th Workshop on Biomedical Natural Language Processing, Berlin, Germany.10.18653/v1/W16-2926 ↗
30. Identifying informative risk factors and predicting bone disease progression via deep belief networks10.1016/j.ymeth.2014.06.011 ↗
31. Liang, Z., Zhang, G., Huang, J.X., and Hu, Q.V. (2014, January 2–5). Deep learning for healthcare decision making with EMRs. Proceedings of the 2014 IEEE International Conference on Bioinformatics and Biomedicine, Belfast, UK.10.1109/BIBM.2014.6999219 ↗
32. Nickerson, P., Tighe, P., Shickel, B., and Rashidi, P. (2016, January 16–20). Deep neural network architectures for forecasting analgesic response. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, Orlando, FL, USA.10.1109/EMBC.2016.7591352 ↗
33. Iozzino, L., Ferrari, C., Large, M., Nielssen, O., and De Girolamo, G. (2015). Prevalence and risk factors of violence by psychiatric acute inpatients: A systematic review and meta-analysis. PLoS ONE, 10.10.1371/journal.pone.0128536 ↗
34. Predictors of violent behavior among acute psychiatric patients: Clinical study10.1111/j.1440-1819.2008.01790.x ↗
35. Examining the role of static and dynamic risk factors in the prediction of inpatient violence: Variable- and person-focused analyses10.1007/s10979-007-9094-8 ↗
36. The antecedents of violence and aggression within psychiatric in-patient settings10.1111/j.1600-0447.2012.01827.x ↗
37. Variables that Predict Assaultiveness in Child Psychiatric Inpatients10.1016/S0002-7138(10)60123-9 ↗
38. Predictors of persistent aggression on the psychiatric inpatient service10.1016/j.comppsych.2013.07.058 ↗
39. A review and meta-analysis of the patient factors associated with psychiatric in-patient aggression10.1111/acps.12053 ↗
40. Prediction of inpatient violence10.1034/j.1600-0447.106.s412.29.x ↗
41. The Meta-Analysis of Clinical Judgment Project: Fifty-Six Years of Accumulated Research on Clinical Versus Statistical Prediction10.1177/0011000005285875 ↗
42. The Relationship Between Level of Training and Accuracy of Violence Risk Assessment10.1176/appi.ps.201200019 ↗
43. Assessing violence risk in general adult psychiatry10.1192/pb.29.4.131 ↗
44. Quinsey, V., Harris, G., Rice, M.E., and Cormier, C. (1998). Violent Offenders: Appraising and Managing Risk, APA Books.10.1037/10304-000 ↗
45. Borum, R., Bartel, P.A., and Forth, A.E. (2005). Structured Assessment of Violence Risk in Youth, PAR.
46. Webster, C.D., Douglas, K.S., Eaves, D., and Hart, S.D. (1997). HCR-20: Assessing Risk of Violence, Version 2, Mental Health, Law and Policy Institute.
47. Use of risk assessment instruments to predict violence and antisocial behaviour in 73 samples involving 24,827 people: Systematic review and meta-analysis10.1136/bmj.e4692 ↗
48. A comparative study of violence risk assessment tools: A systematic review and metaregression analysis of 68 studies involving 25,980 participants10.1016/j.cpr.2010.11.009 ↗
49. The efficacy of violence prediction: A meta-analytic comparison of nine risk assessment tools10.1037/a0020473 ↗
50. The prediction of violence in adult offenders: A meta-analytic comparison of instruments and methods of assessment10.1177/0093854809333610 ↗
51. Standardized risk assessment: Why all the fuss?10.1192/pb.27.6.201 ↗
52. Assessing violence risk and psychopathy in juvenile and adult offenders: A survey of clinical practices10.1177/1073191109359587 ↗
53. Ozomaro, U., Wahlestedt, C., and Nemeroff, C.B. (2013). Personalized medicine in psychiatry: Problems and promises. BMC Med., 11.10.1186/1741-7015-11-132 ↗
54. Menger, V., Scheepers, F., van Wijk, L.M., and Spruit, M. (2017). DEDUCE: A pattern matching method for automatic de-identification of Dutch medical text. Telemat. Inform.10.1016/j.tele.2017.08.002 ↗
55. Bird, S., Klein, E., and Loper, E. (2009). Natural Language Processing with Python, O’Reilly Media, Inc.
56. Lan, M., Tan, C.-L., Low, H.-B., and Sung, S.-Y. (2005, January 10–14). A comprehensive comparative study on term weighting schemes for text categorization with support vector machines. Proceedings of the Special Interest Tracks and Posters of the 14th International Conference on World Wide Web—WWW’05, Chiba, Japan.10.1145/1062745.1062854 ↗
57. Fürnkranz, J. (1998). A Study Using N-Gram Features for Text Categorization, Austrian Research Institute for Artificial Intelligence.
58. Automatic Text Classification: A. Technical Review
59. A comparative study of feature selection and multiclass classification methods for tissue classification based on gene expression10.1093/bioinformatics/bth267 ↗
60. Chiu, B., Crichton, G., Korhonen, A., and Pyysalo, S. (2016, January 12). How to Train good Word Embeddings for Biomedical NLP. Proceedings of the 15th Workshop on Biomedical Natural Language Processing, Berlin, Germany.10.18653/v1/W16-2922 ↗
61. Aggarwal, C.C., and Zhai, C.X. (2013). Mining Text Data, Springer.10.1007/978-1-4614-3223-4 ↗
62. Text Classification and Classifiers: A Survey
63. Automatic classification of Tamil documents using vector space model and artificial neural network10.1016/j.eswa.2009.02.010 ↗
64. Deshpande, V.P., Erbacher, R.F., and Harris, C. (2007, January 20–22). An evaluation of Naïve Bayesian anti-spam filtering techniques. Proceedings of the 2007 IEEE Workshop on Information Assurance, IAW, West Point, NY, USA.10.1109/IAW.2007.381951 ↗
65. Automated Arabic Text Categorization Using SVM and NB
66. On strategies for imbalanced text classification using SVM: A comparative study10.1016/j.dss.2009.07.011 ↗
67. A two-stage feature selection method for text categorization by using information gain, principal component analysis and genetic algorithm10.1016/j.knosys.2011.04.014 ↗
68. Liu, Z., Yang, M., Wang, X., Chen, Q., Tang, B., Wang, Z., and Xu, H. (2017). Entity recognition from clinical texts via recurrent neural network. BMC Med. Inform. Decis. Mak., 17.10.1186/s12911-017-0468-7 ↗
69. Tang, D., Qin, B., and Liu, T. (2015, January 17–21). Document Modeling with Gated Recurrent Neural Network for Sentiment Classification. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.10.18653/v1/D15-1167 ↗
70. Medical Text Classification Using Convolutional Neural Networks
71. Zhang, X., Zhao, J., and LeCun, Y. (arXiv, 2015). Character-level Convolutional Networks for Text Classification, arXiv.
72. Bengio, Y. (arXiv, 2012). Practical recommendations for gradient-based training of deep architectures, arXiv.10.1007/978-3-642-35289-8_26 ↗