A Multi-Channel Uncertainty-Aware Multi-Resolution Network for MR to CT Synthesis
Source abstract
Abstract
Synthesising computed tomography (CT) images from magnetic resonance images (MRI) plays an important role in the field of medical image analysis, both for quantification and diagnostic purposes. Convolutional neural networks (CNNs) have achieved state-of-the-art results in image-to-image translation for brain applications. However, synthesising whole-body images remains largely uncharted territory, involving many challenges, including large image size and limited field of view, complex spatial context, and anatomical differences between images acquired at different times. We propose the use of an uncertainty-aware multi-channel multi-resolution 3D cascade network specifically aiming for whole-body MR to CT synthesis. The Mean Absolute Error on the synthetic CT generated with the MultiResunc network (73.90 HU) is compared to multiple baseline CNNs like 3D U-Net (92.89 HU), HighRes3DNet (89.05 HU) and deep boosted regression (77.58 HU) and shows superior synthesis performance. We ultimately exploit the extrapolation properties of the MultiRes networks on sub-regions of the body.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Image Processing Techniques · Advanced Vision and Imaging · Medical Image Segmentation Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Depterment Medical Physics & Biomedical Engineering, University College London, London WC1E 6BT, UK
School of Biomedical Engineering & Imaging Sciences, King’s College London, London SE1 7EH, UK
Centre for Medical Imaging, University College London, London W1W 7TS, UK
Institute of Nuclear Medicine, University College London, London NW1 2BU, UK
Funding
EPSRC Centre for Doctoral Training in Medical Imaging · Wellcome · Innovative Medicines Initiative
References (23)
Show source-provided references
1. A multi-centre evaluation of eleven clinically feasible brain PET/MRI attenuation correction techniques using a large cohort of patients10.1016/j.neuroimage.2016.12.010 ↗
2. Kläser, K., Markiewicz, P., Ranzini, M., Li, W., Modat, M., Hutton, B.F., Atkinson, D., Thielemans, K., Cardoso, M.J., and Ourselin, S. (2018). Deep boosted regression for MR to CT synthesis. International Workshop on Simulation and Synthesis in Medical Imaging, Springer.10.1007/978-3-030-00536-8_7 ↗
3. Wolterink, J.M., Dinkla, A.M., Savenije, M.H., Seevinck, P.R., van den Berg, C.A., and Išgum, I. (2017). Deep MR to CT synthesis using unpaired data. International Workshop on Simulation and Synthesis in Medical Imaging, Springer.10.1007/978-3-319-68127-6_2 ↗
4. Unpaired whole-body MR to CT synthesis with correlation coefficient constrained adversarial learning
5. Nah, S., Hyun Kim, T., and Mu Lee, K. (2017, January 21–26). Deep multi-scale convolutional neural network for dynamic scene deblurring. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.35 ↗
6. Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Van Der Smagt, P., Cremers, D., and Brox, T. (2015, January 7–13). Flownet: Learning optical flow with convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.316 ↗
7. Eigen, D., Puhrsch, C., and Fergus, R. (2014). Depth map prediction from a single image using a multi-scale deep network. arXiv.
8. Kawahara, J., and Hamarneh, G. (2016). Multi-resolution-tract CNN with hybrid pretrained and skin-lesion trained layers. International Workshop on Machine Learning in Medical Imaging, Springer.10.1007/978-3-319-47157-0_20 ↗
9. Zamzmi, G., Rajaraman, S., and Antani, S. (2020). Accelerating Super-Resolution and Visual Task Analysis in Medical Images. Appl. Sci., 10.10.3390/app10124282 ↗
10. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation10.1016/j.media.2016.10.004 ↗
11. Bragman, F.J., Tanno, R., Eaton-Rosen, Z., Li, W., Hawkes, D.J., Ourselin, S., Alexander, D.C., McClelland, J.R., and Cardoso, M.J. (2018). Uncertainty in multitask learning: Joint representations for probabilistic MR-only radiotherapy planning. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.10.1007/978-3-030-00937-3_1 ↗
12. Exploring uncertainty measures in deep networks for multiple sclerosis lesion detection and segmentation10.1016/j.media.2019.101557 ↗
13. Tanno, R., Worrall, D., Kaden, E., Ghosh, A., Grussu, F., Bizzi, A., Sotiropoulos, S.N., Criminisi, A., and Alexander, D.C. (2019). Uncertainty quantification in deep learning for safer neuroimage enhancement. arXiv.
14. Aleatory or epistemic? Does it matter?10.1016/j.strusafe.2008.06.020 ↗
15. Kendall, A., and Gal, Y. (2017). What uncertainties do we need in bayesian deep learning for computer vision?. arXiv.
16. Reinhold, J.C., He, Y., Han, S., Chen, Y., Gao, D., Lee, J., Prince, J.L., and Carass, A. (2020, January 4). Validating uncertainty in medical image translation. Proceedings of the IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, IA, USA.10.1109/ISBI45749.2020.9098543 ↗
17. Kläser, K., Borges, P., Shaw, R., Ranzini, M., Modat, M., Atkinson, D., Thielemans, K., Hutton, B., Goh, V., and Cook, G. (2020). Uncertainty-Aware Multi-resolution Whole-Body MR to CT Synthesis. International Workshop on Simulation and Synthesis in Medical Imaging, Springer.10.1007/978-3-030-59520-3_12 ↗
18. Çiçek, Ö. (2016). 3D U-Net: Learning dense volumetric segmentation from sparse annotation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.10.1007/978-3-319-46723-8_49 ↗
19. Weighted channel dropout for regularization of deep convolutional neural network
20. Li, W., Wang, G., Fidon, L., Ourselin, S., Cardoso, M.J., and Vercauteren, T. (2017). On the compactness, efficiency, and representation of 3D convolutional networks: Brain parcellation as a pretext task. International Conference on Information Processing in Medical Imaging, Springer.10.1007/978-3-319-59050-9_28 ↗
21. NiftyNet: A deep-learning platform for medical imaging10.1016/j.cmpb.2018.01.025 ↗
22. Attenuation correction synthesis for hybrid PET-MR scanners: Application to brain studies10.1109/TMI.2014.2340135 ↗
23. Fast free-form deformation using graphics processing units10.1016/j.cmpb.2009.09.002 ↗