Recent Advances in Artificial Intelligence-Assisted Ultrasound Scanning
Source abstract
Abstract
Ultrasound (US) is a flexible imaging modality used globally as a first-line medical exam procedure in many different clinical cases. It benefits from the continued evolution of ultrasonic technologies and a well-established US-based digital health system. Nevertheless, its diagnostic performance still presents challenges due to the inherent characteristics of US imaging, such as manual operation and significant operator dependence. Artificial intelligence (AI) has proven to recognize complicated scan patterns and provide quantitative assessments for imaging data. Therefore, AI technology has the potential to help physicians get more accurate and repeatable outcomes in the US. In this article, we review the recent advances in AI-assisted US scanning. We have identified the main areas where AI is being used to facilitate US scanning, such as standard plane recognition and organ identification, the extraction of standard clinical planes from 3D US volumes, and the scanning guidance of US acquisitions performed by humans or robots. In general, the lack of standardization and reference datasets in this field makes it difficult to perform comparative studies among the different proposed methods. More open-access repositories of large US datasets with detailed information about the acquisition are needed to facilitate the development of this very active research field, which is expected to have a very positive impact on US imaging.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Radiomics and Machine Learning in Medical Imaging · Artificial Intelligence in Healthcare and Education · Ultrasound in Clinical Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Family Medicine Department, Centro de Salud de Arroyomolinos, Arroyomolinos, 28939 Madrid, Spain
Advanced Health Technology Department, GMV, Tres Cantos, 28760 Madrid, Spain
Emergency Radiology Department, La Paz University Hospital, 28046 Madrid, Spain
Nuclear Physics Group and IPARCOS, Faculty of Physical Sciences, University Complutense of Madrid, CEI Moncloa, 28040 Madrid, Spain
Health Research Institute of the Hospital Clínico San Carlos (IdISSC), 28040 Madrid, Spain
Funding
Spanish Ministry of Economic Affairs and Digital Transformation
References (72)
Show source-provided references
1. Computer-Based Monitoring of Cardiovascular Functions in Postoperative Patients10.1161/01.CIR.37.4S2.II-68 ↗
2. A History of the Shift Toward Full Computerization of Medicine10.1200/jop.2005.1.2.54 ↗
3. Turing, A.M. (1992). Intelligent Machinery, 1948, 4. Reprinted in Mechanical Intelligence (Collected Works of AM Turing), North-Holland Publishing Co.
4. With an Eye to AI and Autonomous Diagnosis10.1038/s41746-018-0048-y ↗
5. Basics of Deep Learning: A Radiologist’s Guide to Understanding Published Radiology Articles on Deep Learning10.3348/kjr.2019.0312 ↗
6. High-Performance Medicine: The Convergence of Human and Artificial Intelligence10.1038/s41591-018-0300-7 ↗
7. Validity of Machine Learning in Biology and Medicine Increased through Collaborations across Fields of Expertise10.1038/s42256-019-0139-8 ↗
8. A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises10.1109/JPROC.2021.3054390 ↗
9. Deep Learning in Medical Ultrasound Image Analysis: A Review10.1109/ACCESS.2021.3071301 ↗
10. Deep Learning in Medical Ultrasound Analysis: A Review10.1016/j.eng.2018.11.020 ↗
11. Functional Ultrasound Imaging: A New Imaging Modality for Neuroscience10.1016/j.neuroscience.2021.03.005 ↗
12. Perceived Barriers in the Use of Ultrasound in Developing Countries10.1186/s13089-015-0028-2 ↗
13. Maraci, M.A., Napolitano, R., Papageorghiou, A., and Noble, J.A. (2014). Searching for Structures of Interest in an Ultrasound Video Sequence, Springer.10.1007/978-3-319-10581-9_17 ↗
14. Improvement in Breast Tumor Discrimination by Support Vector Machines and Speckle-Emphasis Texture Analysis10.1016/S0301-5629(02)00788-3 ↗
15. Automated Annotation and Quantitative Description of Ultrasound Videos of the Fetal Heart10.1016/j.media.2016.11.006 ↗
16. Classification of Ultrasound Kidney Images Using PCA and Neural Networks
17. Chen, H., Dou, Q., Ni, D., Cheng, J.-Z., Qin, J., Li, S., and Heng, P.-A. (2015). Automatic Fetal Ultrasound Standard Plane Detection Using Knowledge Transferred Recurrent Neural Networks, Springer.10.1007/978-3-319-24553-9_62 ↗
18. Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks10.1109/JBHI.2015.2425041 ↗
19. Ni, D., Li, T., Yang, X., Qin, J., Li, S., Chin, C.-T., Ouyang, S., Wang, T., and Chen, S. (2013). Selective Search and Sequential Detection for Standard Plane Localization in Ultrasound, Springer.10.1007/978-3-642-41083-3_23 ↗
20. Standard Plane Localization in Ultrasound by Radial Component Model and Selective Search10.1016/j.ultrasmedbio.2014.06.006 ↗
21. Rahmatullah, B., Sarris, I., Papageorghiou, A., and Noble, J.A. (2011). Quality Control of Fetal Ultrasound Images: Detection of Abdomen Anatomical Landmarks Using AdaBoost, IEEE.10.1109/ISBI.2011.5872342 ↗
22. Rahmatullah, B., Papageorghiou, A.T., and Noble, J.A. (2012). Integration of Local and Global Features for Anatomical Object Detection in Ultrasound, Springer.10.1007/978-3-642-33454-2_50 ↗
23. Wu, H., Bowers, D.M., Huynh, T.T., and Souvenir, R. (2013). Echocardiogram View Classification Using Low-Level Features, IEEE.10.1109/ISBI.2013.6556584 ↗
24. Agarwal, D., Shriram, K., and Subramanian, N. (2013). Automatic View Classification of Echocardiograms Using Histogram of Oriented Gradients, IEEE.10.1109/ISBI.2013.6556787 ↗
25. Kumar, R., Wang, F., Beymer, D., and Syeda-Mahmood, T. (2009, January 20–25). Echocardiogram View Classification Using Edge Filtered Scale-Invariant Motion Features. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206838 ↗
26. Liver Fibrosis Grade Classification with B-Mode Ultrasound10.1016/S0301-5629(03)01010-X ↗
27. Lei, B., Tan, E.-L., Chen, S., Zhuo, L., Li, S., Ni, D., and Wang, T. (2015). Automatic Recognition of Fetal Facial Standard Plane in Ultrasound Image via Fisher Vector. PloS ONE, 10.10.1371/journal.pone.0121838 ↗
28. Deep Learning-Based Methodology for Recognition of Fetal Brain Standard Scan Planes in 2D Ultrasound Images10.1109/ACCESS.2019.2950387 ↗
29. Cai, Y., Sharma, H., Chatelain, P., and Noble, J.A. (2018). SonoEyeNet: Standardized Fetal Ultrasound Plane Detection Informed by Eye Tracking, IEEE.10.1109/ISBI.2018.8363851 ↗
30. Baumgartner, C.F., Kamnitsas, K., Matthew, J., Smith, S., Kainz, B., and Rueckert, D. (2016). Real-Time Standard Scan Plane Detection and Localisation in Fetal Ultrasound Using Fully Convolutional Neural Networks, Springer.10.1007/978-3-319-46723-8_24 ↗
31. SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound10.1109/TMI.2017.2712367 ↗
32. Chen, Y., Zhang, C., Liu, L., Feng, C., Dong, C., Luo, Y., and Wan, X. (2020). Effective Sample Pair Generation for Ultrasound Video Contrastive Representation Learning. arXiv.
33. Gao, Y., Beriwal, S., Craik, R., Papageorghiou, A.T., and Noble, J.A. (2020). Medical Ultrasound, and Preterm, Perinatal and Paediatric Image Analysis, Springer.
34. A Deep Learning Solution for Automatic Fetal Neurosonographic Diagnostic Plane Verification Using Clinical Standard Constraints10.1016/j.ultrasmedbio.2017.07.013 ↗
35. Evaluation of Deep Convolutional Neural Networks for Automatic Classification of Common Maternal Fetal Ultrasound Planes10.1038/s41598-020-67076-5 ↗
36. Bridge, C.P., and Noble, J.A. (2015). Object Localisation in Fetal Ultrasound Images Using Invariant Features, IEEE.10.1109/ISBI.2015.7163839 ↗
37. Chykeyuk, K., Yaqub, M., and Alison Noble, J. (2013). Class-Specific Regression Random Forest for Accurate Extraction of Standard Planes from 3D Echocardiography, Springer.10.1007/978-3-319-14104-6_6 ↗
38. Lorenz, C., Brosch, T., Ciofolo-Veit, C., Tobias, K., Lefevre, T., Salim, I., Papageorghiou, A.T., Raynaud, C., Roundhill, D., and Rouet, L. (2018). Automated Abdominal Plane and Circumference Estimation in 3D US for Fetal Screening, SPIE.10.1117/12.2292729 ↗
39. Li, Y., Khanal, B., Hou, B., Alansary, A., Cerrolaza, J.J., Sinclair, M., Matthew, J., Gupta, C., Knight, C., and Kainz, B. (2018). Standard Plane Detection in 3d Fetal Ultrasound Using an Iterative Transformation Network, Springer.10.1007/978-3-030-00928-1_45 ↗
40. Li, Y., Cerrolaza, J.J., Sinclair, M., Hou, B., Alansary, A., Khanal, B., Matthew, J., Kainz, B., and Rueckert, D. (2023, March 09). Standard Plane Localisation in 3D Fetal Ultrasound Using Network with Geometric and Image Loss. Available online: https://openreview.net/forum?id=BykcN8siz.
41. Ryou, H., Yaqub, M., Cavallaro, A., Roseman, F., Papageorghiou, A., and Noble, J.A. (2016). Automated 3D Ultrasound Biometry Planes Extraction for First Trimester Fetal Assessment, Springer.10.1007/978-3-319-47157-0_24 ↗
42. Dou, H., Yang, X., Qian, J., Xue, W., Qin, H., Wang, X., Yu, L., Wang, S., Xiong, Y., and Heng, P.-A. (2019). Agent with Warm Start and Active Termination for Plane Localization in 3D Ultrasound, Springer.10.1007/978-3-030-32254-0_33 ↗
43. Searching Collaborative Agents for Multi-Plane Localization in 3d Ultrasound10.1016/j.media.2021.102119 ↗
44. A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
45. Dong, X., and Yang, Y. (2019, January 15–20). Searching for a Robust Neural Architecture in Four Gpu Hours. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA. Available online: https://openaccess.thecvf.com/content_CVPR_2019/papers/Dong_Searching_for_a_Robust_Neural_Architecture_in_Four_GPU_Hours_CVPR_2019_paper.pdf.10.1109/CVPR.2019.00186 ↗
46. Jiang, B., Xu, K., Taylor, R.H., Graham, E., Unberath, M., and Boctor, E.M. (2020). Standard Plane Extraction From 3D Ultrasound With 6-DOF Deep Reinforcement Learning Agent, IEEE.10.1109/IUS46767.2020.9251555 ↗
47. Yu, T.-F., Liu, P., Peng, Y.-L., Liu, J.-Y., Yin, H., and Liu, D.C. (2019). Slice Localization for Three-Dimensional Breast Ultrasound Volume Using Deep Learning. DEStech Trans. Eng. Technol. Res.10.12783/dtetr/icicr2019/30578 ↗
48. Towards Automated Extraction of 2D Standard Fetal Head Planes from 3D Ultrasound Acquisitions: A Clinical Evaluation and Quality Assessment Comparison10.1016/j.radi.2020.11.006 ↗
49. Baum, Z., Hu, Y., and Barratt, D.C. (2020). Medical Ultrasound, and Preterm, Perinatal and Paediatric Image Analysis, Springer.
50. Variational Formulation of Unsupervised Deep Learning for Ultrasound Image Artifact Removal10.1109/TUFFC.2021.3056197 ↗
51. The Effect of Dynamic Range Alterations in the Estimation of Contrast10.1109/TUFFC.2019.2911267 ↗
52. The Generalized Contrast-to-Noise Ratio: A Formal Definition for Lesion Detectability10.1109/TUFFC.2019.2956855 ↗
53. Automatic Quality Assessment for 2D Fetal Sonographic Standard Plane Based on Multitask Learning10.1097/MD.0000000000024427 ↗
54. 3D Freehand Ultrasound without External Tracking Using Deep Learning10.1016/j.media.2018.06.003 ↗
55. Guo, H., Xu, S., Wood, B., and Yan, P. (2020). Sensorless Freehand 3D Ultrasound Reconstruction via Deep Contextual Learning, Springer.10.1007/978-3-030-59716-0_44 ↗
56. Artificial Intelligence for Ultrasonography: Unique Opportunities and Challenges10.14366/usg.20078 ↗
57. Jarosik, P., and Lewandowski, M. (2019). Automatic Ultrasound Guidance Based on Deep Reinforcement Learning, IEEE.10.1109/ULTSYM.2019.8926041 ↗
58. Milletari, F., Birodkar, V., and Sofka, M. (2019). Smart Ultrasound Imaging and Perinatal, Preterm and Paediatric Image Analysis, Springer.
59. Droste, R., Drukker, L., Papageorghiou, A.T., and Noble, J.A. (2020). Automatic Probe Movement Guidance for Freehand Obstetric Ultrasound, Springer.10.1007/978-3-030-59716-0_56 ↗
60. Pan, Y., Cheng, C.-A., Saigol, K., Lee, K., Yan, X., Theodorou, E., and Boots, B. (2017). Agile Autonomous Driving Using End-to-End Deep Imitation Learning. arXiv.10.15607/RSS.2018.XIV.056 ↗
61. Utility of a Deep-Learning Algorithm to Guide Novices to Acquire Echocardiograms for Limited Diagnostic Use10.1001/jamacardio.2021.0185 ↗
62. Toporek, G., Wang, H., Balicki, M., and Xie, H. (2023, March 09). Autonomous Image-Based Ultrasound Probe Positioning via Deep Learning; EasyChair Preprint no. 119. Available online: https://easychair.org/publications/preprint/XmZ9.
63. Mustafa, A.S.B., Ishii, T., Matsunaga, Y., Nakadate, R., Ishii, H., Ogawa, K., Saito, A., Sugawara, M., Niki, K., and Takanishi, A. (2013). Development of Robotic System for Autonomous Liver Screening Using Ultrasound Scanning Device, IEEE.10.1109/ROBIO.2013.6739561 ↗
64. Three-Dimensional Ultrasound Guidance of Autonomous Robotic Breast Biopsy: Feasibility Study10.1016/j.ultrasmedbio.2009.08.014 ↗
65. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
66. Kendall, A., Grimes, M., and Cipolla, R. (2015, January 11–18). Posenet: A convolutional network for real-time 6-dof camera relocalization. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.336 ↗
67. Hase, H., Azampour, M.F., Tirindelli, M., Paschali, M., Simson, W., Fatemizadeh, E., and Navab, N. (2020). Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning, IEEE.10.1109/IROS45743.2020.9340913 ↗
68. Robotic Arm Based Automatic Ultrasound Scanning for Three-Dimensional Imaging10.1109/TII.2018.2871864 ↗
69. Key Challenges for Delivering Clinical Impact with Artificial Intelligence10.1186/s12916-019-1426-2 ↗
70. Computer-Aided Diagnosis System for Thyroid Nodules on Ultrasonography: Diagnostic Performance and Reproducibility Based on the Experience Level of Operators10.1007/s00330-018-5772-9 ↗
71. The “5Es” of Emergency Physician–Performed Focused Cardiac Ultrasound: A Protocol for Rapid Identification of Effusion, Ejection, Equality, Exit, and Entrance10.1111/acem.12652 ↗
72. (2023, March 09). FDA Authorizes Marketing of First Cardiac Ultrasound Software That Uses Artificial Intelligence to Guide User. Feb 2020. FDA news, Available online: https://www.fda.gov/news-events/press-announcements/fda-authorizes-marketing-first-cardiac-ultrasound-software-uses-artificial-intelligence-guide-user.