journal article·2024-07-12Open access

Enhancing Survival Analysis Model Selection through XAI(t) in Healthcare

Francesco Berloco, Pietro Maria Marvulli, Vladimiro Suglia, Simona Colucci, Gaetano Pagano, Lucia Palazzo, Maria Aliani, Giorgio Castellana, Patrizia Guido, Giovanni D’Addio, Vitoantonio Bevilacqua

Applied Sciences· Volume 14· Issue 14

Abstract

Artificial intelligence algorithms have become extensively utilized in survival analysis for high-dimensional, multi-source data. However, due to their complexity, these methods often yield poorly interpretable outcomes, posing challenges in the analysis of several conditions. One of these conditions is obstructive sleep apnea, a sleep disorder characterized by the simultaneous occurrence of comorbidities. Survival analysis provides a potential solution for assessing and categorizing the severity of obstructive sleep apnea, aiding personalized treatment strategies. Given the critical role of time in such scenarios and considering limitations in model interpretability, time-dependent explainable artificial intelligence algorithms have been developed in recent years for direct application to basic Machine Learning models, such as Cox regression and survival random forest. Our work aims to enhance model selection in OSA survival analysis using time-dependent XAI for Machine Learning and Deep Learning models. We developed an end-to-end pipeline, training several survival models and selecting the best performers. Our top models—Cox regression, Cox time, and logistic hazard—achieved good performance, with C-index scores of 0.81, 0.78, and 0.77, and Brier scores of 0.10, 0.12, and 0.11 on the test set. We applied SurvSHAP methods to Cox regression and logistic hazard to investigate their behavior. Although the models showed similar performance, our analysis established that the results of the log hazard model were more reliable and useful in clinical practice compared to those of Cox regression in OSA scenarios.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Obstructive Sleep Apnea Research · Energy Load and Power Forecasting · Machine Learning in Healthcare

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Electrical and Information Engineering (DEI), Politecnico di Bari, Via Orabona 4, 70125 Bari, BA, Italy

Bioengineering Unit of Bari Institute, Istituti Clinici Scientifici Maugeri IRCCS, Via Generale Nicola Bellomo 73/75, 70124 Bari, BA, Italy

Respiratory Rehabilitation Unit of Bari Institute, Istituti Clinici Scientifici Maugeri IRCCS, Via Generale Nicola Bellomo 73/75, 70124 Bari, BA, Italy

Bioengineering Unit of Telese Terme Institute, Istituti Clinici Scientifici Maugeri IRCCS, Via Bagni Vecchi 1, 82037 Telese Terme, BN, Italy

Funding

European Union—Next-Generation EU

References (57)

Show source-provided references
1. Age and comorbidities are crucial predictors of mortality in severe obstructive sleep apnoea syndrome10.1016/j.ejim.2021.04.018 ↗
2. A Deep Survival Learning Approach for Cardiovascular Risk Estimation in Patients with Sleep Apnea10.1109/ACCESS.2022.3231743 ↗
3. Pagano, G., Aliani, M., Genco, M., Coccia, A., Proscia, V., Cesarelli, M., and D’Addio, G. (2022, January 22–24). Rehabilitation outcome in patients with obstructive sleep apnea syndrome using wearable inertial sensor for gait analysis. Proceedings of the 2022 IEEE International Symposium on Medical Measurements and Applications, MeMeA 2022—Conference Proceedings, Messina, Italy.10.1109/MeMeA54994.2022.9856405 ↗
4. Assessment of obstructive sleep apnea-related sleep fragmentation utilizing Deep Learning-based sleep staging from photoplethysmography10.1093/sleep/zsab142 ↗
5. Diagnostic decision support of heart rate turbulence in sleep apnea syndrome
6. Combined unsupervised-supervised Machine Learning for phenotyping complex diseases with its application to obstructive sleep apnea10.1038/s41598-021-84003-4 ↗
7. Machine Learning for survival analysis: A survey10.1145/3214306 ↗
8. A Decision-tree Approach to Stratify DLBCL Risk Based on Stromal and Immune Microenvironment Determinants10.1097/HS9.0000000000000862 ↗
9. Altini, N., Brunetti, A., Mazzoleni, S., Moncelli, F., Zagaria, I., Prencipe, B., Lorusso, E., Buonamico, E., Carpagnano, G.E., and Bavaro, D.F. (2021). Predictive Machine Learning Models and Survival Analysis for COVID-19 Prognosis Based on Hematochemical Parameters. Sensors, 21.10.3390/s21248503 ↗
10. Machine Learning for atrial fibrillation risk prediction in patients with sleep apnea and coronary artery disease10.3389/fcvm.2022.1050409 ↗
11. Wang, M., Greenberg, M., Forkert, N.D., Chekouo, T., Afriyie, G., Ismail, Z., Smith, E.E., and Sajobi, T.T. (2022). Dementia risk prediction in individuals with mild cognitive impairment: A comparison of Cox regression and Machine Learning models. BMC Med. Res. Methodol., 22.10.1186/s12874-022-01754-y ↗
12. Deep Learning-guided adjuvant chemotherapy selection for elderly patients with breast cancer10.1007/s10549-023-07237-y ↗
13. Zaccaria, G.M., Altini, N., Mezzolla, G., Vegliante, M.C., Stranieri, M., Pappagallo, S.A., Ciavarella, S., Guarini, A., and Bevilacqua, V. (2024). SurvIAE: Survival prediction with Interpretable Autoencoders from Diffuse Large B-Cells Lymphoma gene expression data. Comput. Methods Programs Biomed., 244.10.1016/j.cmpb.2023.107966 ↗
14. Detailed Assessment of Sleep Architecture with Deep Learning and Shorter Epoch-to-Epoch Duration Reveals Sleep Fragmentation of Patients with Obstructive Sleep Apnea10.1109/JBHI.2020.3043507 ↗
15. Distributed Analytics For Big Data: A Survey10.1016/j.neucom.2024.127258 ↗
16. Bevilacqua, V., Altini, N., Prencipe, B., Brunetti, A., Villani, L., Sacco, A., Morelli, C., Ciaccia, M., and Scardapane, A. (2021). Lung Segmentation and Characterization in COVID-19 Patients for Assessing Pulmonary Thromboembolism: An Approach Based on Deep Learning and Radiomics. Electronics, 10.10.3390/electronics10202475 ↗
17. Odds ratio, hazard ratio and relative risk
18. SurvSHAP(t): Time-dependent explanations of Machine Learning survival models10.1016/j.knosys.2022.110234 ↗
19. Hospital Length of Stay Prediction Based on Multi-modal Data Towards Trustworthy Human-AI Collaboration in Radiomics
20. Explainable Machine Learning (XAI) for Survival in Bone Marrow Transplantation Trials: A Technical Report10.3390/biomedinformatics3030048 ↗
21. Machine Learning explainability in nasopharyngeal cancer survival using LIME and SHAP10.1038/s41598-023-35795-0 ↗
22. Srinidhi, B., and Bhargavi, M.S. (2023, January 9–10). An XAI Approach to Predictive Analytics of Pancreatic Cancer. Proceedings of the 2023 International Conference on Information Technology (ICIT), Amman, Jordan.
23. Healthcare Analytics A Machine Learning and explainable artificial intelligence approach for predicting the efficacy of hematopoietic stem cell transplant in pediatric patients10.1016/j.health.2023.100170 ↗
24. An Explainable Artificial Intelligence Framework for the Deterioration Risk Prediction of Hepatitis Patients10.1007/s10916-021-01736-5 ↗
25. Lundberg, S.M., and Lee, S.I. (2017, January 4–9). A unified approach to interpreting model predictions. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
26. Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13–17). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’16), New York, NY, USA.10.1145/2939672.2939778 ↗
27. SurvLIME: A method for explaining Machine Learning survival models10.1016/j.knosys.2020.106164 ↗
28. Scrutinio, D. (2024, April 27). Dataset Related to Study: “Age and Comorbidity Are Crucial Predictors of Mortality in Severe Obstructive Sleep Apnoea Syndrome”. Available online: https://zenodo.org/records/4290149.
29. Normal reference values for glomerular filtration rate: What do we really know?10.1093/ndt/gfs265 ↗
30. (2024, April 20). A Healthy Lifestyle—WHO Recommendations. Available online: https://www.who.int/europe/news-room/fact-sheets/item/a-healthy-lifestyle---who-recommendations.
31. Mortality in obstructive sleep apnea syndrome (OSAS) and overlap syndrome (OS): The role of nocturnal hypoxemia and CPAP compliance10.1016/j.sleep.2023.10.011 ↗
32. Increased Prevalence of Sleep-Disordered Breathing in Adults10.1093/aje/kws342 ↗
33. Heilbrunn, E.S., Ssentongo, P., Chinchilli, V.M., Oh, J., and Ssentongo, A.E. (2021). Sudden death in individuals with obstructive sleep apnoea: A systematic review and meta-analysis. BMJ Open Respir. Res., 8.10.1136/bmjresp-2020-000656 ↗
34. Regression Models and Life-Tables10.1111/j.2517-6161.1972.tb00899.x ↗
35. Random Survival Forests10.1214/08-AOAS169 ↗
36. The State of Boosting
37. Support Vector Machines for Survival Analysis with R10.32614/RJ-2018-005 ↗
38. Kvamme, H., Borgan, Ø., and Scheel, I. (2019). Time-to-Event Prediction with Neural Networks and Cox Regression. arXiv.
39. Lee, C., Zame, W., Yoon, J., and Schaar, M. (2018, January 2–7). DeepHit: A Deep Learning Approach to Survival Analysis with Competing Risks. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11842 ↗
40. Katzman, J., Shaham, U., Cloninger, A., Bates, J., Jiang, T., and Kluger, Y. (2018). DeepSurv: Personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Med. Res. Methodol., 18.10.1186/s12874-018-0482-1 ↗
41. A scalable discrete-time survival model for neural networks10.7717/peerj.6257 ↗
42. Eleuteri, A., Aung, M., Taktak, A., Damato, B., and Lisboa, P. (2007, January 22–26). Continuous and Discrete Time Survival Analysis: Neural Network Approaches. Proceedings of the 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Lyon, France.10.1109/IEMBS.2007.4353568 ↗
43. SurvNAM: The Machine Learning survival model explanation10.1016/j.neunet.2021.12.015 ↗
44. Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4 ↗
45. Summary measure of discrimination in survival models based on cumulative/dynamic time-dependent ROC curves10.1177/0962280213515571 ↗
46. Verification of forecasts expressed in terms of probability10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2 ↗
47. Explainable AI models for predicting drop coalescence in microfluidics device10.1016/j.cej.2023.148465 ↗
48. Chen, D., Cheng, S., Hu, J., Kasoar, M., and Arcucci, R. (2024). Explainable Global Wildfire Prediction Models using Graph Neural Networks. arXiv.
49. Altini, N., Puro, E., Taccogna, M.G., Marino, F., De Summa, S., Saponaro, C., Mattioli, E., Zito, F.A., and Bevilacqua, V. (2023). Tumor Cellularity Assessment of Breast Histopathological Slides via Instance Segmentation and Pathomic Features Explainability. Bioengineering, 10.10.3390/bioengineering10040396 ↗
50. Potential value of mitochondrial regulatory pathways in the clinical application of clear cell renal cell carcinoma: A Machine Learning-based study10.1007/s00432-023-05393-8 ↗
51. Zuo, Y., Liu, Q., Li, N., Li, P., Zhang, J., and Song, S. (2023). Optimal 18F-FDG PET/CT radiomics model development for predicting EGFR mutation status and prognosis in lung adenocarcinoma: A multicentric study. Front. Oncol., 13.10.3389/fonc.2023.1173355 ↗
52. Unrestricted permutation forces extrapolation: Variable importance requires at least one more model, or there is no free variable importance10.1007/s11222-021-10057-z ↗
53. Obstructive sleep apnea is associated with impaired renal function in patients with diabetic kidney disease10.1038/s41598-021-85023-w ↗
54. COPD-OSA Overlap Syndrome: Evolving Evidence Regarding Epidemiology, Clinical Consequences, and Management10.1016/j.chest.2017.04.160 ↗
55. Anemia of aging and obstructive sleep apnea10.1007/s11325-010-0326-7 ↗
56. Obstructive sleep apnea and incidence of malignant tumors: A meta-analysis10.1016/j.sleep.2021.05.029 ↗
57. Obesity, obstructive sleep apnea and type 2 diabetes mellitus: Epidemiology and pathophysiologic insights