journal article·2024-04-03Open access

Ultrasound-Based Deep Learning Models Performance versus Expert Subjective Assessment for Discriminating Adnexal Masses: A Head-to-Head Systematic Review and Meta-Analysis

Mariana Lourenço, Teresa Arrufat, Elena Satorres, Sara Maderuelo, Blanca Novillo-Del Álamo, Stefano Guerriero, Rodrigo Orozco, Juan Luis Alcázar

Applied Sciences· Volume 14· Issue 7

Abstract

(1) Background: Accurate preoperative diagnosis of ovarian masses is crucial for optimal treatment and postoperative outcomes. Transvaginal ultrasound is the gold standard, but its accuracy depends on operator skill and technology. In the absence of expert imaging, pattern-based approaches have been proposed. The integration of artificial intelligence, specifically deep learning (DL), shows promise in improving diagnostic precision for adnexal masses. Our meta-analysis aims to evaluate DL’s performance compared to expert evaluation in diagnosing adnexal masses using ultrasound images. (2) Methods: Studies published between 2000 and 2023 were searched in PubMed, Scopus, Cochrane and Web of Science. The study quality was assessed using QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2). Pooled sensitivity and specificity for both methods were estimated and compared. (3) Results: From 1659 citations, we selected four studies to include in this meta-analysis. The mean prevalence of ovarian cancer was 30.6%. The quality of the studies was good with low risk of bias for index and reference tests, but with high risk of bias for patient selection domain. Pooled sensitivity and specificity were 86.0% and 90.0% for DL and 86.0% and 89.0% for expert accuracy (p = 0.9883). (4) Conclusion: We found no significant differences between DL systems and expert evaluations in detecting and differentially diagnosing adnexal masses using ultrasound images.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Ovarian cancer diagnosis and treatment · Cervical Cancer and HPV Research · Endometrial and Cervical Cancer Treatments

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Obstetrics and Gynecology, Hospital Vila Franca de Xira, 2600-009 Lisboa, Portugal

Department of Obstetrics and Gynecology, Hospital de Castellón, 12004 Castellón, Spain

Department of Obstetrics and Gynecology, Hospital Universitario y Politécnico La Fe, 46026 Valencia, Spain

Department of Obstetrics and Gynecology, Universitá di Cagliari, Policlinico Universitario Duilio Casula, 09042 Cagliari, Italy

Department of Obstetrics and Gynecology, Hospital QuirónSalud, 29004 Málaga, Spain

School of Medicine, University of Navarra, 31009 Pamplona, Spain

References (74)

Show source-provided references
1. Use of multiphasic oral contraceptives and hospitalizations of women with functional ovarian cysts in the United States
2. Expectant management of adnexal masses in selected premenopausal women: A prospective observational study10.1002/uog.12369 ↗
3. Risk of complications in patients with conservatively managed ovarian tumours (IOTA5): A 2-year interim analysis of a multicentre, prospective, cohort study10.1016/S1470-2045(18)30837-4 ↗
4. First International Consensus Report on Adnexal Masses: Management Recommendations10.1002/jum.14197 ↗
5. Fertility-Sparing Approaches in Gynecologic Oncology: Role of Imaging in Treatment Planning10.1016/j.rcl.2019.10.006 ↗
6. Epidemiology of epithelial ovarian cancer
7. Cancer statistics, 202010.3322/caac.21590 ↗
8. Ovarian cancer screening and mortality in the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS): A randomized controlled trial10.1016/S0140-6736(15)01224-6 ↗
9. Effect of Hospital Case-Volume on Mortality after Ovarian Cancer Surgery: A Population-Based Retrospective Cohort Study10.1159/000526833 ↗
10. Impact of hospital surgical volume on complete gross resection (CGR) rates following primary debulking surgery for advanced stage epithelial ovarian carcinoma10.1016/j.ygyno.2019.05.016 ↗
11. American College of Obstetricians and Gynecologists’ Committee on Practice Bulletins—Gynecology (2016). Practice Bulletin No. 174: Evaluation and Management of Adnexal Masses. Obstet. Gynecol., 128, e210–e226.10.1097/AOG.0000000000001768 ↗
12. Guideline No. 403: Initial Investigation and Management of Adnexal Masses10.1016/j.jogc.2019.08.044 ↗
13. Confidence of expert ultrasound operators in making a diagnosis of adnexal tumor: Effect on diagnostic accuracy and interobserver agreement10.1002/uog.7335 ↗
14. Ultrasound experience substantially impacts on diagnostic performance and confidence when adnexal masses are classified using pattern recognition10.1159/000265012 ↗
15. Effect of quality of gynaecological ultrasonography on management of patients with suspected ovarian cancer: A randomised controlled trial10.1016/S1470-2045(08)70005-6 ↗
16. External validation of IOTA simple descriptors and simple rules for classifying adnexal masses10.1002/uog.15854 ↗
17. Adnexal masses: What is the role of ultrasonographic imaging?10.1007/s00404-014-3327-0 ↗
18. Transvaginal sonographic characterization of ovarian disease: Evaluation of a new scoring system to predict ovarian malignancy
19. A new scoring system to differentiate benign from malignant adnexal masses10.1067/mob.2003.176 ↗
20. GI-RADS reporting system for ultrasound evaluation of adnexal masses in clinical practice: A prospective multicenter study10.1002/uog.9012 ↗
21. Simple ultrasound-based rules for the diagnosis of ovarian cancer10.1002/uog.5365 ↗
22. O-RADS US Risk Stratification and Management System: A Consensus Guideline from the ACR Ovarian-Adnexal Reporting and Data System Committee10.1148/radiol.2019191150 ↗
23. A systematic review on the use of artificial intelligence in gynecologic imaging. Background, state of the art, and future directions10.1016/j.ygyno.2022.07.024 ↗
24. Logistic models and artificial intelligence in the sonographic assessment of adnexal masses—A systematic review of the literature10.11152/mu-2538 ↗
25. Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology10.1002/uog.22122 ↗
26. Ovarian tumor characterization using 3D ultrasound10.7785/tcrt.2012.500272 ↗
27. Alqasemi, U., Kumavor, P., Aguirre, A., and Zhu, Q. (2012). Recognition algorithm for assisting ovarian cancer diagnosis from coregistered ultrasound and photoacoustic images: Ex vivo study. J. Biomed. Opt., 17.10.1117/1.JBO.17.12.126003 ↗
28. Ovarian tumor characterization and classification: A class of GyneScanTM systems
29. Ovarian tumor characterization and classification using ultrasound—A new online paradigm10.1007/s10278-012-9553-8 ↗
30. GyneScan: An improved online paradigm for screening of ovarian cancer via tissue characterization10.7785/tcrtexpress.2013.600273 ↗
31. Evolutionary algorithm- based classifier parameter tuning for automatic ovarian cancer tissue characterization and classification
32. Pathak, H., and Kulkarni, V. (2015, January 20–22). Identification of ovarian mass through ultrasound images using machine learning techniques. Proceedings of the 2015 IEEE International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), Kolkata, India.10.1109/ICRCICN.2015.7434224 ↗
33. Ultrasound image discrimination between benign and malignant adnexal masses based on a neural network approach10.1016/j.ultrasmedbio.2015.11.014 ↗
34. Martínez-Más, J., Bueno-Crespo, A., Khazendar, S., Remezal-Solano, M., Martínez-Cendán, J.P., Jassim, S., Du, H., Al Assam, H., and Bourne, T. (2019). Evaluation of machine learning methods with Fourier transform features for classifying ovarian tumors based on ultrasound images. PLoS ONE, 14.10.1371/journal.pone.0219388 ↗
35. Artificial intelligence in ovarian cancer diagnosis10.21873/anticanres.14482 ↗
36. Application of deep convolutional neural networks for discriminating benign, borderline, and malignant serous ovarian tumors from ultrasound images10.3389/fonc.2021.770683 ↗
37. The Adoption of radiomics and machine learning improves the diagnostic processes of women with ovarian masses (the AROMA pilot study)10.1007/s40477-020-00503-5 ↗
38. Ștefan, P.A., Lupean, R.A., Mihu, C.M., Lebovici, A., Oancea, M.D., Hîțu, L., Duma, D., and Csutak, C. (2021). Ultrasonography in the diagnosis of adnexal lesions: The role of texture analysis. Diagnostics, 11.10.3390/diagnostics11050812 ↗
39. An evaluation of the effectiveness of image-based texture features extracted from static B mode ultrasound images in distinguishing between benign and malignant ovarian masses10.1177/0161734621998091 ↗
40. Ultrasound image analysis using deep neural networks for discriminating between benign and malignant ovarian tumors: Comparison with expert subjective assessment10.1002/uog.23530 ↗
41. Establishment and verification of logistic regression model for qualitative diagnosis of ovarian cancer based on MRI and ultrasound signs
42. Deep learning prediction of ovarian malignancy at US compared with O-RADS and expert assessment10.1148/radiol.211367 ↗
43. Li, J., Chen, Y., Zhang, M., Zhang, P., He, K., Yan, F., Li, J., Xu, H., Burkhoff, D., and Luo, Y. (2022). A Deep Learning Model System for Diagnosis and Management of Adnexal Masses. Cancers, 14.10.3390/cancers14215291 ↗
44. Deep learning-enabled pelvic ultrasound images for accurate diagnosis of ovarian cancer in China: A retrospective, multicentre, diagnostic study10.1016/S2589-7500(21)00278-8 ↗
45. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews10.1136/bmj.n71 ↗
46. QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies10.7326/0003-4819-155-8-201110180-00009 ↗
47. Measuring inconsistency in meta-analyses10.1136/bmj.327.7414.557 ↗
48. ESGO/ISUOG/ IOTA/ESGE Consensus Statement on pre-operative diagnosis of ovarian tumors10.1136/ijgc-2021-002565 ↗
49. Subjective assessment versus ultrasound models to diagnose ovarian cancer: A systematic review and meta-analysis10.1016/j.ejca.2016.01.007 ↗
50. A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer10.1111/j.1471-0528.1990.tb02448.x ↗
51. Risk of Ovarian Malignancy Algorithm versus Risk Malignancy Index-I for Preoperative Assessment of Adnexal Masses: A Systematic Review and Meta-Analysis10.1159/000501681 ↗
52. Logistic regression model to distinguish between the benign and malignant adnexal mass before surgery: A multicenter study by the International Ovarian Tumor Analysis Group10.1200/JCO.2005.01.7632 ↗
53. Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: Prospective multicentre diagnostic study10.1136/bmj.g5920 ↗
54. Gynecologic imaging reporting and data system: A new proposal for classifying adnexal masses on the basis of sonographic findings10.7863/jum.2009.28.3.285 ↗
55. Presurgical diagnosis of adnexal tumours using mathematical models and scoring systems: A systematic review and meta-analysis10.1093/humupd/dmt059 ↗
56. IOTA Simple Rules for the differential diagnosis of ovarian adnexal masses: Systematic review and meta-analysis
57. Value of Assessment of Different Neoplasias in the Adnexa in the Differential Diagnosis of Malignant Ovarian Tumor and Benign Ovarian Tumor: A Meta-analysis10.1016/j.ultrasmedbio.2022.02.001 ↗
58. ADNEX risk prediction model for diagnosis of ovarian cancer: Systematic review and meta-analysis of external validation studies10.1136/bmjmed-2023-000817 ↗
59. The diagnostic performance of the Gynecologic Imaging Reporting and Data System (GI-RADS) in adnexal masses10.21037/atm-20-5170 ↗
60. Gynecologic Imaging and Reporting Data System for classifying adnexal masses
61. Vara, J., Manzour, N., Chacón, E., López-Picazo, A., Linares, M., Pascual, M.Á., Guerriero, S., and Alcázar, J.L. (2022). Ovarian Adnexal Reporting Data System (O-RADS) for Classifying Adnexal Masses: A Systematic Review and Meta-Analysis. Cancers, 14.10.3390/cancers14133151 ↗
62. O-RADS US: A Systematic Review and Meta-Analysis of Category-specific Malignancy Rates10.1148/radiol.223269 ↗
63. Systematic Review and Meta-Analysis of O-RADS Ultrasound and O-RADS MRI for Risk Assessment of Ovarian and Adnexal Lesions10.2214/AJR.22.28396 ↗
64. Contributions of Artificial Intelligence Reported in Obstetrics and Gynecology Journals: Systematic Review10.2196/35465 ↗
65. Jost, E., Kosian, P., Jimenez Cruz, J., Albarqouni, S., Gembruch, U., Strizek, B., and Recker, F. (2023). Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology. J. Clin. Med., 12.10.3390/jcm12216833 ↗
66. Sonographic prediction of malignancy in adnexal masses using an artificial neural network10.1111/j.1471-0528.1999.tb08080.x ↗
67. Artificial neural network models for the preoperative discrimination between malignant and benign adnexal masses10.1046/j.1469-0705.1999.13010017.x ↗
68. Predicting ovarian malignancy: Application of artificial neural networks to transvaginal and color Doppler flow US10.1148/radiology.210.2.r99fe18399 ↗
69. Artificial neural network computer prediction of ovarian malignancy in women with adnexal masses10.1016/j.ijgo.2005.01.034 ↗
70. Artificial intelligence performance in image-based ovarian cancer identification: A systematic review and meta-analysis
71. AI diagnostic performance based on multiple imaging modalities for ovarian tumor: A systematic review and meta-analysis10.3389/fonc.2023.1133491 ↗
72. Deep learning approaches for COVID-19 detection based on chest X-ray images10.1016/j.eswa.2020.114054 ↗
73. Deep learning radiomics of ultrasonography for differentiating sclerosing adenosis from breast cancer10.3233/CH-221608 ↗
74. Computer aid screening of COVID-19 using X-ray and CT scan images: An inner comparison