Towards Trustful Machine Learning for Antimicrobial Therapy Using an Explainable Artificial Intelligence Dashboard
Source abstract
Abstract
The application of machine learning (ML) in healthcare has surged, yet its adoption in high-stakes clinical domains, like the Intensive Care Unit (ICU), remains low. This gap is largely driven by a lack of clinician trust in AI decision support. Explainable AI (XAI) techniques aim to address this by explaining how an AI reaches its decisions, thereby improving transparency. However, rigorous evaluation of XAI methods in clinical settings is lacking. Therefore, we evaluated the perceived explainability of a dashboard incorporating three XAI methods for an ML model that predicts piperacillin plasma concentrations. The dashboard was evaluated by seven ICU clinicians using five distinct patient cases. We assessed the interpretation and perceived explainability of each XAI component through a targeted survey. The overall dashboard received a median score of seven out of ten for completeness of explainability, with Ceteris Paribus profiles identified as the most preferred XAI method. Our findings provide a practical framework for evaluating XAI in critical care, offering crucial insights into clinician preferences that can guide the future development and implementation of trustworthy AI in the ICU.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Explainable Artificial Intelligence (XAI) · Machine Learning in Healthcare · Imbalanced Data Classification Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Internal Medicine and Pediatrics, Faculty of Medicine and Health Sciences, Ghent University, 9052 Ghent, Belgium
Department of Intensive Care Medicine, Ghent University Hospital, 9000 Ghent, Belgium
IDLab, Ghent University-imec, 9052 Ghent, Belgium
Funding
FWO Junior Research project HEROI2C · Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” program · Research Foundation Flanders
References (41)
Show source-provided references
1. A Systematic Review of Human–Computer Interaction and Explainable Artificial Intelligence in Healthcare With Artificial Intelligence Techniques10.1109/ACCESS.2021.3127881 ↗
2. (2023, November 15). Available online: https://ezra.com/en-gb.
3. Zhang, J., Whebell, S., Gallifant, J., Budhdeo, S., Mattie, H., Lertvittayakumjorn, P., Lopez, M.P.A., Tiangco, B.J., Gichoya, J.W., and Ashrafian, H. (2021). An interactive dashboard to track themes, development maturity, and global equity in clinical artificial intelligence research. medRxiv.10.1101/2021.11.23.21266758 ↗
4. Artificial intelligence-supported screen reading versus standard double reading in the Mammography Screening with Artificial Intelligence trial (MASAI): A clinical safety analysis of a randomised, controlled, non-inferiority, single-blinded, screening accuracy study10.1016/S1470-2045(23)00298-X ↗
5. An evaluation framework for Health Information Systems: Human, organization and technology-fit factors (HOT-fit)10.1016/j.ijmedinf.2007.08.011 ↗
6. De Corte, T., Van Maele, L., Dietvorst, J., Verhaeghe, J., Vandendriessche, A., De Neve, N., Vanderhaeghen, S., Dumoulin, A., Temmerman, W., and Dewulf, B. (J. Crit. Care, 2025). Exploring Preconditions for the Implementation of Artificial Intelligence-based Clinical Decision Support Systems in the Intensive Care Unit—A Multicentric Mixed Methods Study, J. Crit. Care, under review.
7. Machine intelligence in healthcare—perspectives on trustworthiness, explainability, usability, and transparency10.1038/s41746-020-0254-2 ↗
8. Tonekaboni, S., Joshi, S., McCradden, M.D., and Goldenberg, A. (2019). What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use. arXiv.
9. Pierce, R.L., Van Biesen, W., Van Cauwenberge, D., Decruyenaere, J., and Sterckx, S. (2022). Explainability in medicine in an era of AI-based clinical decision support systems. Front. Genet., 13.10.3389/fgene.2022.903600 ↗
10. Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13–17). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939778 ↗
11. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems 30, Curran Associates, Inc.
12. A systematic review of Explainable Artificial Intelligence models and applications: Recent developments and future trends10.1016/j.dajour.2023.100230 ↗
13. A prospective randomized clinical trial for measuring radiology study reporting time on Artificial Intelligence-based detection of intracranial hemorrhage in emergent care head CT
14. Zhang, W.-f., Li, D.-h., Wei, Q.-j., Ding, D.-y., Meng, L.-h., Wang, Y.-l., Zhao, X.-y., and Chen, Y.-x. (2022). The Validation of Deep Learning-Based Grading Model for Diabetic Retinopathy. Front. Med., 9.10.3389/fmed.2022.839088 ↗
15. A Real-Time Automated Patient Screening System for Clinical Trials Eligibility in an Emergency Department: Design and Evaluation10.2196/14185 ↗
16. Norrie, C. (2021). Explainable AI Techniques for Sepsis Diagnosis: Evaluating LIME and SHAP Through a User Study. [Master’s Thesis, University of Skovde].
17. Verhaeghe, J., Dhaese, S.A.M., De Corte, T., Vander Mijnsbrugge, D., Aardema, H., Zijlstra, J.G., Verstraete, A.G., Stove, V., Colin, P., and Ongenae, F. (2022). Development and evaluation of uncertainty quantifying machine learning models to predict piperacillin plasma concentrations in critically ill patients. BMC Med. Inform. Decis. Mak., 22.10.1186/s12911-022-01970-y ↗
18. Linardatos, P., Papastefanopoulos, V., and Kotsiantis, S. (2020). Explainable AI: A Review of Machine Learning Interpretability Methods. Entropy, 23.10.3390/e23010018 ↗
19. Zhou, J., Gandomi, A.H., Chen, F., and Holzinger, A. (2021). Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics. Electronics, 10.10.3390/electronics10050593 ↗
20. Anchors: High-Precision Model-Agnostic Explanations
21. Supersparse linear integer models for optimized medical scoring systems10.1007/s10994-015-5528-6 ↗
22. Sharchilev, B., Ustinovsky, Y., Serdyukov, P., and de Rijke, M. (2018). Finding Influential Training Samples for Gradient Boosted Decision Trees. arXiv.
23. dalex: Responsible Machine Learning with Interactive Explainability and Fairness in Python
24. Mothilal, R.K., Sharma, A., and Tan, C. (2020, January 27–30). Explaining machine learning classifiers through diverse counterfactual explanations. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Barcelona, Spain.10.1145/3351095.3372850 ↗
25. Generating multi-level explanations for process outcome predictions10.1016/j.engappai.2023.106678 ↗
26. An explainable decision support system for predictive process analytics10.1016/j.engappai.2023.105904 ↗
27. (2023, November 27). Knowledge Centre Data & Society HC-XAI Evaluation Inventory. Available online: https://data-en-maatschappij.ai/en/publications/xai-decision-tool-2021.
28. A complete and multifaceted overview of antibiotic use and infection diagnosis in the intensive care unit: Results from a prospective four-year registration10.1186/s13054-018-2178-7 ↗
29. Prevalence and Outcomes of Infection Among Patients in Intensive Care Units in 201710.1001/jama.2020.2717 ↗
30. World Health Organization (2023, October 17). Ten Health Issues WHO Will Tackle This Year, Available online: https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019.
31. Roberts, J.A., De Waele, J.J., Dimopoulos, G., Koulenti, D., Martin, C., Montravers, P., Rello, J., Rhodes, A., Starr, T., and Wallis, S.C. (2012). DALI: Defining Antibiotic Levels in Intensive care unit patients: A multi-centre point of prevalence study to determine whether contemporary antibiotic dosing for critically ill patients is therapeutic. BMC Infect. Dis., 12.10.1186/1471-2334-12-152 ↗
32. Antimicrobial therapeutic drug monitoring in critically ill adult patients: A Position Paper10.1007/s00134-020-06050-1 ↗
33. Beta-Lactam Antibiotic Therapeutic Drug Monitoring in Critically Ill Patients: A Systematic Review and Meta-Analysis10.1093/cid/ciac506 ↗
34. Verhaeghe, J., De Corte, T., De Waele, J.J., Ongenae, F., and Van Hoecke, S. (2024, January 17–19). Designing a Pharmacokinetic Machine Learning Model for Optimizing Beta-Lactam Antimicrobial Dosing in Critically Ill Patients. Proceedings of the ICMHI ’24: 2024 8th International Conference on Medical and Health Informatics, New York, NY, USA.10.1145/3673971.3673982 ↗
35. Grinberg, M. (2018). Flask Web Development: Developing Web Applications with Python, O’Reilly Media, Inc.
36. Meta (2023, February 16). React: A JavaScript Library for Building User Interfaces. Available online: https://legacy.reactjs.org/.
37. Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., and Wortman Vaughan, J. (2020, January 25–30). Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning. Proceedings of the CHI ’20: 2020 CHI Conference on Human Factors in Computing Systems, New York, NY, USA.10.1145/3313831.3376219 ↗
38. Gkatzia, D., Lemon, O., and Rieser, V. (2016). Natural Language Generation enhances human decision-making with uncertain information. arXiv.10.18653/v1/P16-2043 ↗
39. One Explanation Does Not Fit All10.1007/s13218-020-00637-y ↗
40. Werner, C. (April, January 30). Explainable AI through Rule-based Interactive Conversation. Proceedings of the EDBT/ICDT Workshops, Copenhagen, Denmark.
41. Ardito, C., Lanzilotti, R., Malizia, A., Petrie, H., Piccinno, A., Desolda, G., and Inkpen, K. (2021). Making SHAP Rap: Bridging Local and Global Insights Through Interaction and Narratives. Human-Computer Interaction—INTERACT 2021, Proceedings of the 18th IFIP TC 13 International Conference, Bari, Italy, 30 August–3 September 2021, Springer. Lecture Notes in Computer Science.10.1007/978-3-030-85610-6 ↗