journal article·2023-01-12Open access

Reverse-Net: Few-Shot Learning with Reverse Teaching for Deformable Medical Image Registration

Xin Zhang, Tiejun Yang, Xiang Zhao, Aolin Yang

Applied Sciences· Volume 13· Issue 2

Abstract

Multimodal medical image registration has an important role in monitoring tumor growth, radiotherapy, and disease diagnosis. Deep-learning-based methods have made great progress in the past few years. However, its success depends on large training datasets, and the performance of the model decreases due to overfitting and poor generalization when only limited data are available. In this paper, a multimodal medical image registration framework based on few-shot learning is proposed, named reverse-net, which can improve the accuracy and generalization ability of the network by using a few segmentation labels. Firstly, we used the border enhancement network to enhance the ROI (region of interest) boundaries of T1 images to provide high-quality data for the subsequent pixel alignment stage. Secondly, through a coarse registration network, the T1 image and T2 image were roughly aligned. Then, the pixel alignment network generated more smooth deformation fields. Finally, the reverse teaching network used the warped T1 segmentation labels and warped images generated by the deformation field to teach the border enhancement network more structural knowledge. The performance and generalizability of our model have been evaluated on publicly available brain datasets including the MRBrainS13DataNii-Pro, SRI24, CIT168, and OASIS datasets. Compared with VoxelMorph, the reverse-net obtained performance improvements of 4.36% in DSC on the publicly available MRBrainS13DataNii-Pro dataset. On the unseen dataset OASIS, the reverse-net obtained performance improvements of 4.2% in DSC compared with VoxelMorph, which shows that the model can obtain better generalizability. The promising performance on dataset CIT168 indicates that the model is practicable.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Medical Image Segmentation Techniques · Advanced Neural Network Applications · Medical Imaging and Analysis

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Information Science and Engineering, Henan University of Technology, Zhengzhou 450001, China

School of Artificial Intelligence and Big Data, Henan University of Technology, Zhengzhou 450001, China

Key Laboratory of Grain Information Processing and Control (HAUT), Ministry of Education, Zhengzhou 450001, China

Henan Key Laboratory of Grain Photoelectric Detection and Control (HAUT), Zhengzhou 450001, China

Funding

key specialized research and development program of Henan Province · Open Fund Project of Key Laboratory of Grain Information Processing & Control · Innovative Funds Plan of Henan University of Technology

References (46)

Show source-provided references
1. Symmetric diffeomorphic image registration with CrossCorrelation: Evaluating automated labeling of elderly and neurodegenerative brain10.1016/j.media.2007.06.004 ↗
2. Image matching as a diffusion process: An analogy with Maxwell’s demons10.1016/S1361-8415(98)80022-4 ↗
3. A fast inverse consistent deformable image registration method based on symmetric optical flow computation10.1088/0031-9155/53/21/017 ↗
4. Deep reinforcement learning in medical imaging: A literature review10.1016/j.media.2021.102193 ↗
5. Real-time deep pose estimation with geodesic loss for Image-to-Template rigid registration10.1109/TMI.2018.2866442 ↗
6. Yan, P., Xu, S., Rastinehad, A.R., and Wood, B.J. (2018). Adversarial image registration with application for MR and TRUS image fusion. International Workshop on Machine Learning in Medical Imaging, Springer.10.1007/978-3-030-00919-9_23 ↗
7. He, Y., Li, T., Yang, G., and Kong, Y. (2020, January 23–28). L Deep complementary joint model for complex scene registration and few-shot segmentation on medical images. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58523-5_45 ↗
8. Anatomy-guided multimodal registration by learning segmentation without ground truth: Application to intraprocedural CBCT/MR liver segmentation and registration10.1016/j.media.2021.102041 ↗
9. Hering, A., Kuckertz, S., Heldmann, S., and Heinrich, M.P. (2019). Enhancing Label-Driven Deep Deformable Image Registration with Local Distance Metrics for State-of-the-Art Cardiac Motion Tracking. Bildverarbeitung für die Medizin, Springer.10.1007/978-3-658-25326-4_69 ↗
10. MATR: Multimodal Medical Image Fusion via Multiscale Adaptive Transformer10.1109/TIP.2022.3193288 ↗
11. Deformable MR-CT image registration using an unsupervised dual-channel network for neurosurgical guidance10.1016/j.media.2021.102292 ↗
12. Zhang, J. (2018). Inverse-Consistent deep networks for unsupervised deformable image registration. arXiv.
13. Mahapatra, D., Antony, B., Sedai, S., and Garnavi, R. (2018, January 4–7). Deformable medical image registration using generative adversarial networks. Proceedings of the IEEE International Symposium on Biomedical Imaging, Washington, DC, USA.10.1109/ISBI.2018.8363845 ↗
14. De Vos, B.D., Berendsen, F.F., Viergever, M.A., Staring, M., and Isgum, I. (2017, January 14). End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network. Proceedings of the Computer Vision and Pattern Recognition, Québec City, QC, Canada.10.1007/978-3-319-67558-9_24 ↗
15. Sun, L., and Zhang, S. (2018, January 16–20). Deformable MRI-Ultrasound Registration Using 3D Convolutional Neural Network. Proceedings of the Simulation, Image Processing, and Ultrasound Systems for Assisted Diagnosis and Navigation, Granada, Spain.10.1007/978-3-030-01045-4_18 ↗
16. Generalizing from a Few Examples: A Survey on Few-Shot Learning
17. Medela, A., Picon, A., Saratxaga, C.L., Belar, O., Cabezón, V., Cicchi, R., Bilbao, R., and Glover, B. (2019, January 8–11). Few Shot Learning in histopathological images:Reducing the need of labeled data on biological datasets. Proceedings of the International Symposium on Biomedical Imaging, Venice, Italy.10.1109/ISBI.2019.8759182 ↗
18. Zhao, A., Balakrishnan, G., Durand, F., Guttag, J.V., and Dalca, A.V. (2019, January 16–20). Data augmentation using learned transformations for One-Shot medical image segmentation. Proceedings of the Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00874 ↗
19. Puch, S., Sanchez, I., and Rowe, M. (2019, January 13–17). Few-shot Learning with deep triplet networks for brain imaging modality recognition. Proceedings of the Medical Image Computing and Computer Assisted Intervention Society, Shenzhen, China.10.1007/978-3-030-33391-1_21 ↗
20. Few-Shot Learning for Deformable Medical Image Registration With Perception-Correspondence Decoupling and Reverse Teaching10.1109/JBHI.2021.3095409 ↗
21. Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2016, January 27–30). Spatial transformer networks. Proceedings of the Conference on Computer Vision and Pattern Recognition, Montreal, QC, Canada.
22. VoxelMorph: A learning framework for deformable medical image registration10.1109/TMI.2019.2897538 ↗
23. Ronneberger, O.F.P., and Brox, T. (2015, January 5–9). U-Net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
24. TransDIR: Deformable imaging registration network based on transformer to improve the feature extraction ability10.1002/mp.15420 ↗
25. TransMorph: Transformer for unsupervised medical image registration10.1016/j.media.2022.102615 ↗
26. GraformerDIR: Graph convolution transformer for deformable image registration10.1016/j.compbiomed.2022.105799 ↗
27. A Review of the Deep Learning Methods for Medical Images Super Resolution Problems10.1016/j.irbm.2020.08.004 ↗
28. Real-time estimation of hospital discharge using fuzzy radial basis function network and electronic health record data
29. Siebert, J.P., Goatman, K.A., and Sloan, J.M. (2018, January 19–21). Learning Rigid Image Registration—Utilizing Convolutional Neural Networks for Medical Image Registration. Proceedings of the International Conference on on Bioimaging, Funchal, Portugal.
30. Liu, Q., and Leung, H. (2017, January 10–13). Tensor-based descriptor for image registration via unsupervised network. Proceedings of the International Conference on Information Fusion, Xi’an, China.10.23919/ICIF.2017.8009689 ↗
31. Intensity-based image registration by minimizing residual complexity10.1109/TMI.2010.2053043 ↗
32. MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration10.1016/j.media.2012.05.008 ↗
33. A full migration BBO algorithm with enhanced population quality bounds for multimodal biomedical image registration10.1016/j.asoc.2020.106335 ↗
34. Medical image registration using Edgeworth-based approximation of Mutual Information10.1016/j.irbm.2013.12.004 ↗
35. Dual-core steered non-rigid registration for multi-modal images via bi-directional image synthesis10.1016/j.media.2017.05.004 ↗
36. A Kernel Correlation-Based Approach to Adaptively Acquire Local Features for Learning 3D Point Clouds10.1016/j.cad.2022.103196 ↗
37. Pedestrian alignment network for large-scale person re-Identification10.1109/TCSVT.2018.2873599 ↗
38. An overview of medical image registration methods
39. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Dalca, A.V., and Guttag, J. (2018, January 18–22). An unsupervised learning model for deformable medical image registration. Proceedings of the Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00964 ↗
40. Mok, T.C.W., and Chung, A.C.S. (2020, January 16–20). Fast symmetric diffeomorphic image registration with convolutional neural networks. Proceedings of the Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00470 ↗
41. MRBrainS Challenge: Online Evaluation Framework for Brain Image Segmentation in 3T MRI Scans10.1155/2015/813696 ↗
42. Open Access Series of Imaging Studies (OASIS): Cross-sectional MRI data in young, middle aged, nondemented, and demented older adults10.1162/jocn.2007.19.9.1498 ↗
43. A high-resolution probabilistic in vivo atlas of human subcortical brain nuclei10.1038/sdata.2018.63 ↗
44. The SRI24 multichannel atlas of normal adult human brain structure10.1002/hbm.20906 ↗
45. Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces10.1016/j.media.2019.07.006 ↗
46. A coarse-to-fine deformable transformation framework for unsupervised multi-contrast MR image registration with dual consistency constraint10.1109/TMI.2021.3059282 ↗