journal article·2022-09-29Open access

The Impact of Image Acquisition Parameters and ComBat Harmonization on the Predictive Performance of Radiomics: A Renal Cell Carcinoma Model

Abdalla Ibrahim, Lin Lu, Hao Yang, Oguz Akin, Lawrence H. Schwartz, Binsheng Zhao

Applied Sciences· Volume 12· Issue 19

Abstract

Radiomics, one of the potential methods for developing clinical biomarker, is one of the exponentially growing research fields. In addition to its potential, several limitations have been identified in this field, and most importantly the effects of variations in imaging parameters on radiomic features (RFs). In this study, we investigate the potential of RFs to predict overall survival in patients with clear cell renal cell carcinoma, as well as the impact of ComBat harmonization on the performance of RF models. We assessed the robustness of the results by performing the analyses a thousand times. Publicly available CT scans of 179 patients were retrospectively collected and analyzed. The scans were acquired using different imaging vendors and parameters in different medical centers. The performance was calculated by averaging the metrics over all runs. On average, the clinical model significantly outperformed the radiomic models. The use of ComBat harmonization, on average, did not significantly improve the performance of radiomic models. Hence, the variability in image acquisition and reconstruction parameters significantly affect the performance of radiomic models. The development of radiomic specific harmonization techniques remain a necessity for the advancement of the field.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Radiomics and Machine Learning in Medical Imaging · Advanced X-ray and CT Imaging · Renal cell carcinoma treatment

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Radiology, Columbia University Irving Medical Center, New York, NY 10032, USA

Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY 10065, USA

Funding

National Institutes of Health

References (51)

Show source-provided references
1. Radiomics: Extracting More Information from Medical Images Using Advanced Feature Analysis10.1016/j.ejca.2011.11.036 ↗
2. Radiomics: Images Are More than Pictures, They Are Data10.1148/radiol.2015151169 ↗
3. Radiomics for Classification of Lung Cancer Histological Subtypes Based on Nonenhanced Computed Tomography10.1016/j.acra.2018.10.013 ↗
4. Yao, Z., Dong, Y., Wu, G., Zhang, Q., Yang, D., Yu, J.-H., and Wang, W.-P. (2018). Preoperative Diagnosis and Prediction of Hepatocellular Carcinoma: Radiomics Analysis Based on Multi-Modal Ultrasound Images. BMC Cancer, 18.10.1186/s12885-018-5003-4 ↗
5. The Role of Preoperative Computed Tomography Radiomics in Distinguishing Benign and Malignant Tumors of the Parotid Gland10.3389/fonc.2021.634452 ↗
6. Granzier, R.W.Y., Ibrahim, A., Primakov, S.P., Samiei, S., van Nijnatten, T.J.A., de Boer, M., Heuts, E.M., Hulsmans, F.-J., Chatterjee, A., and Lambin, P. (2021). MRI-Based Radiomics Analysis for the Pretreatment Prediction of Pathologic Complete Tumor Response to Neoadjuvant Systemic Therapy in Breast Cancer Patients: A Multicenter Study. Cancers, 13.10.3390/cancers13102447 ↗
7. Changes in Magnetic Resonance Imaging Radiomic Features in Response to Androgen Deprivation Therapy in Patients with Intermediate- and High-Risk Prostate Cancer10.1016/j.clon.2021.12.020 ↗
8. Radiomics Response Signature for Identification of Metastatic Colorectal Cancer Sensitive to Therapies Targeting EGFR Pathway10.1093/jnci/djaa017 ↗
9. A Fully-Automatic Multiparametric Radiomics Model: Towards Reproducible and Prognostic Imaging Signature for Prediction of Overall Survival in Glioblastoma Multiforme10.1038/s41598-017-14753-7 ↗
10. Pre-Treatment CT Radiomics to Predict 3-Year Overall Survival Following Chemoradiotherapy of Esophageal Cancer10.1080/0284186X.2018.1486039 ↗
11. Applications and Limitations of Radiomics10.1088/0031-9155/61/13/R150 ↗
12. Radiomics for Precision Medicine: Current Challenges, future Prospects, and the Proposal of a New Framework10.1016/j.ymeth.2020.05.022 ↗
13. What Are Biomarkers?10.1097/COH.0b013e32833ed177 ↗
14. Ibrahim, A., Refaee, T., Leijenaar, R.T.H., Primakov, S., Hustinx, R., Mottaghy, F.M., Woodruff, H.C., Maidment, A.D.A., and Lambin, P. (2021). The Application of a Workflow Integrating the Variable Reproducibility and Harmonizability of Radiomic Features on a Phantom Dataset. PLoS ONE, 16.10.1371/journal.pone.0251147 ↗
15. Repeatability and Reproducibility Study of Radiomic Features on a Phantom and Human Cohort10.1038/s41598-021-81526-8 ↗
16. Ibrahim, A., Refaee, T., Primakov, S., Barufaldi, B., Acciavatti, R.J., Granzier, R.W.Y., Hustinx, R., Mottaghy, F.M., Woodruff, H.C., and Wildberger, J.E. (2021). The Effects of In-Plane Spatial Resolution on CT-Based Radiomic Features’ Stability with and without ComBat Harmonization. Cancers, 13.10.3390/cancers13081848 ↗
17. Reproducibility of CT Radiomic Features within the Same Patient: Influence of Radiation Dose and CT Reconstruction Settings10.1148/radiol.2019190928 ↗
18. Understanding Sources of Variation to Improve the Reproducibility of Radiomics10.3389/fonc.2021.633176 ↗
19. Radiomic Feature Reproducibility in Contrast-Enhanced CT of the Pancreas Is Affected by Variabilities in Scan Parameters and Manual Segmentation10.1007/s00330-019-06381-8 ↗
20. Mali, S.A., Ibrahim, A., Woodruff, H.C., Andrearczyk, V., Müller, H., Primakov, S., Salahuddin, Z., Chatterjee, A., and Lambin, P. (2021). Making Radiomics More Reproducible across Scanner and Imaging Protocol Variations: A Review of Harmonization Methods. J. Pers. Med., 11.10.3390/jpm11090842 ↗
21. Li, Y., Ammari, S., Balleyguier, C., Lassau, N., and Chouzenoux, E. (2021). Impact of Preprocessing and Harmonization Methods on the Removal of Scanner Effects in Brain MRI Radiomic Features. Cancers, 13.10.3390/cancers13123000 ↗
22. Stability of Radiomic Features of Apparent Diffusion Coefficient (ADC) Maps for Locally Advanced Rectal Cancer in Response to Image Pre-Processing10.1016/j.ejmp.2019.04.009 ↗
23. Effects of Contrast-Enhancement, Reconstruction Slice Thickness and Convolution Kernel on the Diagnostic Performance of Radiomics Signature in Solitary Pulmonary Nodule10.1038/srep34921 ↗
24. Impact of CT Convolution Kernel on Robustness of Radiomic Features for Different Lung Diseases and Tissue Types10.1259/bjr.20200947 ↗
25. Refaee, T., Salahuddin, Z., Widaatalla, Y., Primakov, S., Woodruff, H.C., Hustinx, R., Mottaghy, F.M., Ibrahim, A., and Lambin, P. (2022). CT Reconstruction Kernels and the Effect of Pre- and Post-Processing on the Reproducibility of Handcrafted Radiomic Features. J. Pers. Med., 12.10.3390/jpm12040553 ↗
26. Reproducibility of F18-FDG PET Radiomic Features for Different Cervical Tumor Segmentation Methods, Gray-Level Discretization, and Reconstruction Algorithms10.1002/acm2.12170 ↗
27. Influence of Gray Level Discretization on Radiomic Feature Stability for Different CT Scanners, Tube Currents and Slice Thicknesses: A Comprehensive Phantom Study10.1080/0284186X.2017.1351624 ↗
28. Repeatability of Multiparametric Prostate MRI Radiomics Features10.1038/s41598-019-45766-z ↗
29. Adjusting Batch Effects in Microarray Expression Data Using Empirical Bayes Methods10.1093/biostatistics/kxj037 ↗
30. Uncontrolled Confounders May Lead to False or Overvalued Radiomics Signature: A Proof of Concept Using Survival Analysis in a Multicenter Cohort of Kidney Cancer10.3389/fonc.2021.638185 ↗
31. (2013). Cancer Genome Atlas Research Network Comprehensive Molecular Characterization of Clear Cell Renal Cell Carcinoma. Nature, 499, 43–49.10.1038/nature12222 ↗
32. Nomograms for Predicting Long-Term Overall Survival and Disease-Specific Survival of Patients with Clear Cell Renal Cell Carcinoma10.2147/OTT.S171881 ↗
33. Overall Survival Prediction in Renal Cell Carcinoma Patients Using Computed Tomography Radiomic and Clinical Information10.1007/s10278-021-00500-y ↗
34. Radiomics-Based Machine Learning Model to Predict Risk of Death within 5-Years in Clear Cell Renal Cell Carcinoma Patients10.1016/j.compbiomed.2020.104135 ↗
35. Development and Validation of a Radiomic Nomogram for Predicting the Prognosis of Kidney Renal Clear Cell Carcinoma10.3389/fonc.2021.613668 ↗
36. The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository10.1007/s10278-013-9622-7 ↗
37. Kirby, J. (2022, June 10). TCGA-KIRC. Available online: https://wiki.cancerimagingarchive.net/display/Public/TCGA-KIRC.
38. Harmonization of Multi-Site Diffusion Tensor Imaging Data10.1016/j.neuroimage.2017.08.047 ↗
39. Nonenhancing Component of Clear Cell Renal Cell Carcinoma on Computed Tomography Correlates with Tumor Necrosis and Stage and Serves as a Size-Independent Prognostic Biomarker10.1097/RCT.0000000000000877 ↗
40. Efficient Screening of Nonnormal Regression Models10.2307/2530022 ↗
41. Zar, J.H. (2005). Spearman Rank Correlation. Encyclopedia of Biostatistics, Wiley Online Library.10.1002/0470011815.b2a15150 ↗
42. Smith, L. (2004). Cox Regression Model, Louisiana State University.
44. Rank Transformations and the Power of the Student T Test and Welch T’test for Non-Normal Populations with Unequal Variances10.1037/h0078850 ↗
45. Team, R.C. (2000). R Language Definition, R foundation for statistical computing.
46. Gandrud, C. (2013). Reproducible Research with R and R Studio, CRC Press.10.1201/b15100 ↗
47. The Mayo Clinic Stage, Size, Grade and Necrosis (ssign) Score for Clear Cell Renal Cell Carcinoma: External Validation by a Single European Centre Applying10.1016/S1569-9056(08)60832-5 ↗
48. Reproducibility of Radiomics for Deciphering Tumor Phenotype with Imaging10.1038/srep23428 ↗
49. Intrinsic Dependencies of CT Radiomic Features on Voxel Size and Number of Gray Levels
50. Voxel Size and Gray Level Normalization of CT Radiomic Features in Lung Cancer10.1038/s41598-018-28895-9 ↗
51. Learning from Scanners: Bias Reduction and Feature Correction in Radiomics