journal article·2024-08-01Open access

DeepSarc-US: A Deep Learning Framework for Assessing Sarcopenia Using Ultrasound Images

Bahareh Behboodi, Jeremy Obrand, Jonathan Afilalo, Hassan Rivaz

Applied Sciences· Volume 14· Issue 15

Abstract

Sarcopenia, the age-related loss of skeletal muscle mass, is a core component of frailty that is associated with functional decline and adverse health events in older adults. Unfortunately, the available tools to diagnose sarcopenia are often inaccessible or not user-friendly for clinicians. Point-of-care ultrasound (US) is a promising tool that has been used to image the quadriceps muscle and measure its thickness (QMT) as a diagnostic criterion for sarcopenia. This measurement can be challenging for clinicians, especially when performed at the bedside using handheld systems or phased-array probes not designed for this use case. In this paper, we sought to automate this measurement using deep learning methods to improve its accuracy, reliability, and speed in the hands of untrained clinicians. In the proposed framework, which aids in better training, particularly when limited data are available, convolutional and transformer-based deep learning models with generic or data-driven pre-trained weights were compared. We evaluated regression (QMT as a continuous output in cm) and classification (QMT as an ordinal output in 0.5 cm bins) approaches, and in the latter, activation maps were generated to interpret the anatomical landmarks driving the model predictions. Finally, we evaluated a segmentation approach to derive QMT. The results showed that both transformer-based models and convolutional neural networks benefit from the proposed framework in estimating QMT. Additionally, the activation maps highlighted the interface between the femur bone and the quadriceps muscle as a key anatomical landmark for accurate predictions. The proposed framework is a pivotal step to enable the application of US-based measurement of QMT in large-scale clinical studies seeking to validate its diagnostic performance for sarcopenia, alone or with ancillary criteria assessing muscle quality or strength. We believe that implementing the proposed framework will empower clinicians to conveniently diagnose sarcopenia in clinical settings and accordingly personalize the care of older patients, leading to improved patient outcomes and a more efficient allocation of healthcare resources.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Nutrition and Health in Aging · Body Composition Measurement Techniques · Frailty in Older Adults

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Electrical and Computer Engineering, Concordia University, Montreal, QC H3G 1M8, Canada

Division of Experimental Medicine, McGill University, Montreal, QC H3A 0C2, Canada

Division of Cardiology and Centre for Clinical Epidemiology, Jewish General Hospital, McGill University, Montreal, QC H3A 0C2, Canada

School of Health, Concordia University, Montreal, QC H3G 1M8, Canada

Funding

Mitacs through the Mitacs Accelerate program · The Natural Sciences and Engineering Research Council of Canada (NSERC) · Canadian Institutes of Health Research · Fonds de Recherche du Québec en Santé

References (56)

Show source-provided references
1. Frailty consensus: A call to action10.1016/j.jamda.2013.03.022 ↗
2. Frailty assessment in the cardiovascular care of older adults10.1016/j.jacc.2013.09.070 ↗
3. Frailty: Implications for clinical practice and public health10.1016/S0140-6736(19)31786-6 ↗
4. Understanding frailty: A qualitative study of European healthcare policy-makers’ approaches to frailty screening and management10.1136/bmjopen-2017-018653 ↗
5. Older adults in the cardiac intensive care unit: Factoring geriatric syndromes in the management, prognosis, and process of care: A scientific statement from the American Heart Association10.1161/CIR.0000000000000741 ↗
6. If you cannot measure frailty, you cannot improve it10.1016/j.jchf.2018.12.015 ↗
7. Measurement and interpretation of handgrip strength for research on sarcopenia and osteoporosis10.11005/jbm.2020.27.2.85 ↗
8. Prognostic value of handgrip strength in older adults undergoing cardiac surgery10.1016/j.cjca.2021.08.016 ↗
9. Muscle mass and mortality after cardiac transplantation10.1097/TP.0000000000002311 ↗
10. Sarcopenia: A useful paradigm for physical frailty10.1016/j.eurger.2013.02.009 ↗
11. Psoas muscle area and length of stay in older adults undergoing cardiac operations10.1016/j.athoracsur.2016.09.005 ↗
12. Sarcopenia and frailty: New challenges for clinical practice10.7861/clinmedicine.16-5-455 ↗
13. Frailty index as a predictor of mortality: A systematic review and meta-analysis10.1093/ageing/afx162 ↗
14. Church, S., Rogers, E., Rockwood, K., and Theou, O. (2020). A scoping review of the Clinical Frailty Scale. BMC Geriatr., 20.10.1186/s12877-020-01801-7 ↗
15. Abdominal musculature segmentation and surface prediction from CT using deep learning for sarcopenia assessment10.1016/j.diii.2020.04.011 ↗
16. Sarcopenia in Cardiac Surgery: Dual X-ray Absorptiometry Study from the McGill Frailty Registry10.1016/j.ahj.2021.04.008 ↗
17. Sarcopenia: Revised European consensus on definition and diagnosis10.1093/ageing/afy169 ↗
18. Muscle ultrasound: Present state and future opportunities10.1002/mus.27081 ↗
19. Echo intensity as an indicator of skeletal muscle quality: Applications, methodology, and future directions10.1007/s00421-020-04556-6 ↗
20. The role of ultrasound as a diagnostic tool for sarcopenia
21. Deep-Learning-Based Color Doppler Ultrasound Image Feature in the Diagnosis of Elderly Patients with Chronic Heart Failure Complicated with Sarcopenia10.1155/2021/2603842 ↗
22. Pintelas, E., Livieris, I.E., Barotsis, N., Panayiotakis, G., and Pintelas, P. (2021, January 25–27). An Autoencoder Convolutional Neural Network Framework for Sarcopenia Detection Based on Multi-frame Ultrasound Image Slices. Proceedings of the IFIP International Conference on Artificial Intelligence Applications and Innovations, Crete, Greece.10.1007/978-3-030-79150-6_17 ↗
23. Sobral, C., Silva, J.S., André, A., and Santos, J.B. (2024, July 23). Sarcopenia Diagnosis: Deep Transfer Learning versus Traditional Machine Learning. Available online: https://recpad2021.uevora.pt/wp-content/uploads/2020/10/RECPAD_2020_paper_2.pdf.
24. Marzola, F., van Alfen, N., Doorduin, J., and Meiburger, K.M. (2021). Deep learning segmentation of transverse musculoskeletal ultrasound images for neuromuscular disease assessment. Comput. Biol. Med., 135.10.1016/j.compbiomed.2021.104623 ↗
25. Liu, Z., Mao, H., Wu, C.Y., Feichtenhofer, C., Darrell, T., and Xie, S. (2022, January 18–24). A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01167 ↗
26. Hassanien, M.A., Singh, V.K., Puig, D., and Abdel-Nasser, M. (2022). Predicting breast tumor malignancy using deep ConvNeXt radiomics and quality-based score pooling in ultrasound sequences. Diagnostics, 12.10.3390/diagnostics12051053 ↗
27. Kim, K., Macruz, F., Wu, D., Bridge, C., McKinney, S., Al Saud, A.A., Sharaf, E., Pely, A., Danset, P., and Duffy, T. (2023). Point-of-care AI-assisted stepwise ultrasound pneumothorax diagnosis. Phys. Med. Biol., 68.10.1088/1361-6560/acfb70 ↗
28. A novel wavelet-transform-based convolution classification network for cervical lymph node metastasis of papillary thyroid carcinoma in ultrasound images10.1016/j.compmedimag.2023.102298 ↗
29. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16 × 16 words: Transformers for image recognition at scale. arXiv.
30. He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. (2022, January 18–24). Masked autoencoders are scalable vision learners. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01553 ↗
31. Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives10.1016/j.media.2023.102762 ↗
32. Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., and Dong, L. (2022, January 18–24). Swin transformer v2: Scaling up capacity and resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01170 ↗
33. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00986 ↗
34. Gheflati, B., and Rivaz, H. (2022, January 11–15). Vision transformers for classification of breast ultrasound images. Proceedings of the 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Scotland, UK.10.1109/EMBC48229.2022.9871809 ↗
35. Li, L., Wu, Z., Liu, J., Wang, L., Jin, Y., Jiang, P., Feng, J., and Wu, M. (2022, January 6–8). Cross-Attention Based Multi-Scale Feature Fusion Vision Transformer For Breast Ultrasound Image Classification. Proceedings of the 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Las Vegas, NV, USA.10.1109/BIBM55620.2022.9994966 ↗
36. Sun, J., Wu, B., Zhao, T., Gao, L., Xie, K., Lin, T., Sui, J., Li, X., Wu, X., and Ni, X. (2023). Classification for thyroid nodule using ViT with contrastive learning in ultrasound images. Comput. Biol. Med., 152.10.1016/j.compbiomed.2022.106444 ↗
37. Zhang, J., Chen, Y., and Liu, P. (2022, January 2–4). Automatic Recognition of Standard Liver Sections Based on Vision-Transformer. Proceedings of the 2022 IEEE 16th International Conference on Anti-Counterfeiting, Security, and Identification (ASID), Xiamen, China.10.1109/ASID56930.2022.9995936 ↗
38. Liu, X., and Almekkawy, M. (2022, January 10–13). Ultrasound Super Resolution using Vision Transformer with Convolution Projection Operation. Proceedings of the 2022 IEEE International Ultrasonics Symposium (IUS), Venice, Italy.10.1109/IUS54386.2022.9958771 ↗
39. Muhtaseb, R., and Yaqub, M. (2022, January 18–22). EchoCoTr: Estimation of the Left Ventricular Ejection Fraction from Spatiotemporal Echocardiography. Proceedings of the Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore. Proceedings, Part IV.10.1007/978-3-031-16440-8_36 ↗
40. Multilevel structure-preserved GAN for domain adaptation in intravascular ultrasound analysis10.1016/j.media.2022.102614 ↗
41. A VGG attention vision transformer network for benign and malignant classification of breast ultrasound images10.1002/mp.15852 ↗
42. Dilated transformer: Residual axial attention for breast ultrasound image segmentation10.21037/qims-22-33 ↗
43. Liljequist, D., Elfving, B., and Skavberg Roaldsen, K. (2019). Intraclass correlation—A discussion and demonstration of basic features. PLoS ONE, 14.10.1371/journal.pone.0219854 ↗
44. A guideline of selecting and reporting intraclass correlation coefficients for reliability research10.1016/j.jcm.2016.02.012 ↗
45. ImageNet Large Scale Visual Recognition Challenge10.1007/s11263-015-0816-y ↗
46. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
47. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
48. Models matter, so does training: An empirical study of cnns for optical flow estimation10.1109/TPAMI.2019.2894353 ↗
49. A comprehensive survey of continual learning: Theory, method and application10.1109/TPAMI.2024.3367329 ↗
50. Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., and Brox, T. (2017, January 21–26). Flownet 2.0: Evolution of optical flow estimation with deep networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.179 ↗
51. Kingma, D., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
52. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
53. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22–29). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.74 ↗
54. Chefer, H., Gur, S., and Wolf, L. (2021, January 19–25). Transformer interpretability beyond attention visualization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00084 ↗
55. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany. Proceedings, Part III 18.
56. Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009, January 14–18). Curriculum learning. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada.10.1145/1553374.1553380 ↗