journal article·2022-11-19Open access

A Deep Learning Approach to Upscaling “Low-Quality” MR Images: An In Silico Comparison Study Based on the UNet Framework

Rishabh Sharma, Panagiotis Tsiamyrtzis, Andrew G. Webb, Ioannis Seimenis, Constantinos Loukas, Ernst Leiss, Nikolaos V. Tsekos

Applied Sciences· Volume 12· Issue 22

Abstract

MR scans of low-gamma X-nuclei, low-concentration metabolites, or standard imaging at very low field entail a challenging tradeoff between resolution, signal-to-noise, and acquisition duration. Deep learning (DL) techniques, such as UNets, can potentially be used to improve such “low-quality” (LQ) images. We investigate three UNets for upscaling LQ MRI: dense (DUNet), robust (RUNet), and anisotropic (AUNet). These were evaluated for two acquisition scenarios. In the same-subject High-Quality Complementary Priors (HQCP) scenario, an LQ and a high quality (HQ) image are collected and both LQ and HQ were inputs to the UNets. In the No Complementary Priors (NoCP) scenario, only the LQ images are collected and used as the sole input to the UNets. To address the lack of same-subject LQ and HQ images, we added data from the OASIS-1 database. The UNets were tested in upscaling 1/8, 1/4, and 1/2 undersampled images for both scenarios. As manifested by non-statically significant differences of matrices, also supported by subjective observation, the three UNets upscaled images equally well. This was in contrast to mixed effects statistics that clearly illustrated significant differences. Observations suggest that the detailed architecture of these UNets may not play a critical role. As expected, HQCP substantially improves upscaling with any of the UNets. The outcomes support the notion that DL methods may have merit as an integral part of integrated holistic approaches in advancing special MRI acquisitions; however, primary attention should be paid to the foundational step of such approaches, i.e., the actual data collected.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Seismic Imaging and Inversion Techniques · Medical Imaging Techniques and Applications · Image and Signal Denoising Methods

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Medical Robotics and Imaging Lab, Department of Computer Science, University of Houston, Houston, TX 77004, USA

Department of Mechanical Engineering, Politecnico di Milano, 20156 Milan, Italy

C.J. Gorter Center for High Field MRI, Department of Radiology, Leiden University Medical Center, P.O. Box 9600, 2333 ZA Leiden, The Netherlands

Laboratory of Medical Physics, National and Kapodistrian University of Athens, 11527 Athens, Greece

Department of Computer Science, University of Houston, Houston, TX 77004, USA

Funding

National Science Foundation

References (63)

Show source-provided references
1. Signal-to-Noise Efficiency in Magnetic Resonance Imaging10.1118/1.596503 ↗
2. Contrast, Resolution, and Detectability in MR Imaging10.1097/00004728-199103000-00021 ↗
4. Super-Resolution Methods in MRI: Can They Improve the Trade-off between Resolution, Signal-to-Noise Ratio, and Acquisition Time?10.1002/mrm.24187 ↗
5. Undersampled Projection Reconstruction Applied to MR Angiography10.1002/(SICI)1522-2594(200001)43:1<91::AID-MRM11>3.0.CO;2-4 ↗
6. Sparse MRI: The Application of Compressed Sensing for Rapid MR Imaging10.1002/mrm.21391 ↗
7. A Brief Review of Parallel Magnetic Resonance Imaging10.1007/s00330-003-1992-7 ↗
8. Encoding and Reconstruction in Parallel MRI10.1002/nbm.1042 ↗
9. Chemical Shift Imaging: A Review10.2214/ajr.146.5.971 ↗
10. Low-field MRI: An MR Physics Perspective10.1002/jmri.26637 ↗
11. X-nuclei Imaging: Current State, Technical Challenges, and Future Directions10.1002/jmri.26780 ↗
12. Pham, C.-H., Ducournau, A., Fablet, R., and Rousseau, F. (2017, January 18–21). Brain MRI Super-Resolution Using Deep 3D Convolutional Networks. Proceedings of the 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), Melbourne, Australia.10.1109/ISBI.2017.7950500 ↗
13. Chen, Y., Shi, F., Christodoulou, A.G., Xie, Y., Zhou, Z., and Li, D. (2018). Efficient and accurate MRI super-resolution using a generative adversarial network and 3D multi-level densely connected network. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer. LNCS.10.1007/978-3-030-00928-1_11 ↗
14. Super-Resolution in Magnetic Resonance Imaging: A Review10.1002/cmr.a.21249 ↗
15. Deep MR Brain Image Super-Resolution Using Spatio-Structural Priors10.1109/TIP.2019.2942510 ↗
16. VolumeNet: A Lightweight Parallel Network for Super-Resolution of MR and CT Volumetric Data10.1109/TIP.2021.3076285 ↗
17. Variable-Density Single-Shot Fast Spin-Echo MRI with Deep Learning Reconstruction by Using Variational Networks10.1148/radiol.2018180445 ↗
18. A Review of Wavelet Denoising in MRI and Ultrasound Brain Imaging10.2174/157340506776930665 ↗
19. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising10.1109/TIP.2017.2662206 ↗
20. Brief Review of Image Denoising Techniques10.1186/s42492-019-0016-7 ↗
21. Iqbal, Z., Nguyen, D., Hangel, G., Motyka, S., Bogner, W., and Jiang, S. (2019). Super-Resolution 1H Magnetic Resonance Spectroscopic Imaging Utilizing Deep Learning. Front. Oncol., 9.10.3389/fonc.2019.01010 ↗
22. Hu, X., Naiel, M.A., Wong, A., Lamm, M., and Fieguth, P. (2019, January 16–17). RUNet: A Robust UNet Architecture for Image Super-Resolution. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA.10.1109/CVPRW.2019.00073 ↗
23. Lin, H., Figini, M., Tanno, R., Blumberg, S.B., Kaden, E., Ogbole, G., Brown, B.J., D’Arco, F., Carmichael, D.W., and Lagunju, I. (2019). Deep learning for low-field to high-field mr: Image quality transfer with probabilistic decimation simulator. International Workshop on Machine Learning for Medical Image Reconstruction, Springer.10.1007/978-3-030-33843-5_6 ↗
24. Deep Learning Single-Frame and Multiframe Super-Resolution for Cardiac MRI10.1148/radiol.2020192173 ↗
25. Chatterjee, S., Sarasaen, C., Rose, G., Nürnberger, A., and Speck, O. (2022). DDoS-UNet: Incorporating Temporal Information Using Dynamic Dual-Channel UNet for Enhancing Super-Resolution of Dynamic MRI. arXiv.
26. Chatterjee, S., Sciarra, A., Dunnwald, M., Mushunuri, R.V., Podishetti, R., Rao, R.N., Gopinath, G.D., Oeltze-Jafra, S., Speck, O., and Nurnberger, A. (2021, January 23–27). ShuffleUNet: Super Resolution of Diffusion-Weighted MRIs Using Deep Learning. Proceedings of the 2021 29th European Signal Processing Conference (EUSIPCO), Dublin, Ireland.10.23919/EUSIPCO54536.2021.9615963 ↗
27. Ding, P.L.K., Li, Z., Zhou, Y., and Li, B. (2019, January 19–21). Deep Residual Dense U-Net for Resolution Enhancement in Accelerated MRI Acquisition. Proceedings of the Medical Imaging 2019: Image Processing, San Diego, California, USA.10.1117/12.2513158 ↗
28. Nasrin, S., Alom, M.Z., Burada, R., Taha, T.M., and Asari, V.K. (2019, January 15–19). Medical Image Denoising with Recurrent Residual U-Net (R2U-Net) Base Auto-Encoder. Proceedings of the 2019 IEEE National Aerospace and Electronics Conference (NAECON), Dayton, OH, USA.10.1109/NAECON46414.2019.9057834 ↗
29. Koonjoo, N., Zhu, B., Bagnall, G.C., Bhutto, D., and Rosen, M.S. (2021). Boosting the Signal-to-Noise of Low-Field MRI with Deep Learning Image Reconstruction. Sci Rep., 11.10.1038/s41598-021-87482-7 ↗
30. Image Super-Resolution Using Progressive Generative Adversarial Networks for Medical Image Analysis10.1016/j.compmedimag.2018.10.005 ↗
31. Sanchez, I., and Vilaplana, V. (2018). Brain MRI Super-Resolution Using 3D Generative Adversarial Networks. arXiv.
32. MRI Super-Resolution With Ensemble Learning and Complementary Priors10.1109/TCI.2020.2964201 ↗
33. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.10.1007/978-3-319-24574-4_28 ↗
34. Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., and Ronneberger, O. (2016, January 17–21). 3D U-Net: Learning dense volumetric segmentation from sparse annotation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention—MICCAI 2016, Athens, Greece.10.1007/978-3-319-46723-8_49 ↗
35. Sharma, R., Eick, C.F., and Tsekos, N.V. (2020, January 26–28). Myocardial Infarction Segmentation in Late Gadolinium Enhanced MRI Images Using Data Augmentation and Chaining Multiple U-Net. Proceedings of the 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE), Cincinnati, Ohio, USA.10.1109/BIBE50027.2020.00165 ↗
36. Soomro, T.A., Afifi, A.J., Gao, J., Hellwich, O., Paul, M., and Zheng, L. (2018, January 10–13). Strided U-Net Model: Retinal Vessels Segmentation Using Dice Loss. Proceedings of the 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia.10.1109/DICTA.2018.8615770 ↗
37. Chen, C., Zhou, K., Wang, Z., and Xiao, R. (2022). Generative Consistency for Semi-Supervised Cerebrovascular Segmentation from TOF-MRA. IEEE Trans. Med. Imaging, 1.10.1109/TMI.2022.3184675 ↗
38. An Effective Deep Neural Network for Lung Lesions Segmentation from COVID-19 CT Images10.1109/TII.2021.3059023 ↗
39. Rapid Whole-Heart CMR with Single Volume Super-Resolution10.1186/s12968-020-00651-x ↗
40. Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., and van Walsum, T. (2019). No New-Net. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Springer International Publishing.10.1007/978-3-030-11723-8 ↗
41. Distribution of Brain Sodium Accumulation Correlates with Disability in Multiple Sclerosis: A Cross-Sectional 23Na MR Imaging Study10.1148/radiol.12112680 ↗
42. In Vivo T1 and T2 Relaxation Time Maps of Brain Tissue, Skeletal Muscle, and Lipid Measured in Healthy Volunteers at 50 MT10.1002/mrm.29009 ↗
43. Pinheiro, J.C., and Bates, D.M. (2000). Mixed-Effects Models in S and S-PLUS, Springer.10.1007/978-1-4419-0318-1 ↗
44. A Review of Medical Image Data Augmentation Techniques for Deep Learning Applications10.1111/1754-9485.13261 ↗
45. Shin, H.-C., Tenenholtz, N.A., Rogers, J.K., Schwarz, C.G., Senjem, M.L., Gunter, J.L., Andriole, K.P., and Michalski, M. (2018). Medical Image Synthesis for Data Augmentation and Anonymization Using Generative Adversarial Networks. International Workshop on Simulation and Synthesis in Medical Imaging, Springer.10.1007/978-3-030-00536-8_1 ↗
46. CT Prostate Segmentation Based on Synthetic MRI-aided Deep Attention Fully Convolution Network10.1002/mp.13933 ↗
47. Head and Neck Multi-organ Auto-segmentation on CT Images Aided by Synthetic MRI10.1002/mp.14378 ↗
48. Lei, Y., Wang, T., Tian, S., Dong, X., Jani, A.B., Schuster, D., Curran, W.J., Patel, P., Liu, T., and Yang, X. (2020). Male Pelvic Multi-Organ Segmentation Aided by CBCT-Based Synthetic MRI. Phys. Med. Biol., 65.10.1088/1361-6560/ab63bb ↗
49. Open Access Series of Imaging Studies (OASIS): Cross-Sectional MRI Data in Young, Middle Aged, Nondemented, and Demented Older Adults10.1162/jocn.2007.19.9.1498 ↗
50. Mzoughi, H., Njeh, I., ben Slima, M., ben Hamida, A., Mhiri, C., and Ben Mahfoudh, K. (2019). Denoising and Contrast-Enhancement Approach of Magnetic Resonance Imaging Glioblastoma Brain Tumors. J. Med. Imaging, 6.10.1117/1.JMI.6.4.044002 ↗
51. Can Virtual Contrast Enhancement in Brain MRI Replace Gadolinium?10.1097/RLI.0000000000000583 ↗
52. Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
53. Ioffe, S., and Szegedy, C. (2015, January 6–11). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on Machine Learning, Lille, France.
54. Nair, V., and Hinton, G.E. (2010, January 21–24). Rectified Linear Units Improve Restricted Boltzmann Machines. Proceedings of the International Conference on Machine Learning, Haifa, Israel.
55. Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv.
56. Deep Learning10.1038/nature14539 ↗
57. Backpropagation Applied to Handwritten Zip Code Recognition10.1162/neco.1989.1.4.541 ↗
58. Shi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D., and Wang, Z. (2016, January 27–30). Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.207 ↗
59. Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21–26). Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
60. Image Super-Resolution Using Deep Convolutional Networks10.1109/TPAMI.2015.2439281 ↗
61. ResViT: Residual Vision Transformers for Multimodal Medical Image Synthesis10.1109/TMI.2022.3167808 ↗
62. Liang, J., Cao, J., Sun, G., Zhang, K., van Gool, L., and Timofte, R. (2021, January 11–17). SwinIR: Image Restoration Using Swin Transformer. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada.10.1109/ICCVW54120.2021.00210 ↗
63. Yang, F., Yang, H., Fu, J., Lu, H., and Guo, B. (2020, January 13–19). Learning Texture Transformer Network for Image Super-Resolution. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00583 ↗