journal article·2026-04-09Open access

Bidirectional T1–T2 Brain MRI Synthesis Using a Fusion U-Net Transformer for Real-World Clinical Data

Zeynep Cantemir, Hacer Karacan, Emetullah Cindil, Burak Kalafat

Applied Sciences· Volume 16· Issue 8

Abstract

Obtaining multiple MRI contrasts for each patient prolongs scan acquisition time, increases healthcare costs, and may not always be feasible due to patient specific constraints. Deep learning-based MRI contrast synthesis offers a potential solution, yet most existing approaches are evaluated on preprocessed public benchmarks that do not reflect real-world clinical variability. In this study, we propose a fusion U-Net transformer framework for bidirectional T1-weighted ↔ T2-weighted brain MRI synthesis trained and evaluated exclusively on retrospectively acquired clinical data. The proposed architecture integrates multiscale convolutional feature extraction with axial attention mechanisms and a transformer bottleneck for efficient global context modeling. A fusion refinement block is incorporated to mitigate skip connection artifacts. An adversarial training strategy with the least squares GAN objective and a hybrid loss combining L1 reconstruction and structural similarity (SSIM) is employed to promote both pixel-level accuracy and perceptual fidelity. The model is evaluated using SSIM and PSNR metrics alongside qualitative expert assessment conducted by two board-certified radiologists. For both synthesis directions, the framework achieves competitive quantitative performance against baseline models under the challenging conditions of clinical data. Expert evaluation confirms high anatomical fidelity and clinically acceptable image quality across both synthesis directions. These results indicate that the proposed framework represents a promising approach for multi-contrast MRI synthesis in clinically heterogeneous data environments.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Generative Adversarial Networks and Image Synthesis · Functional Brain Connectivity Studies · Advanced Neural Network Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Graduate School of Natural and Applied Sciences, Gazi University, 06500 Ankara, Turkey

Department of Computer Engineering, Faculty of Engineering, Gazi University, 06570 Ankara, Turkey

Department of Radiology, Faculty of Medicine, Gazi University, 06500 Ankara, Turkey

References (40)

Show source-provided references
1. Rising use of diagnostic medical imaging in a large integrated health system10.1377/hlthaff.27.6.1491 ↗
2. MR pulse sequences: What every radiologist wants to know but is afraid to ask10.1148/rg.262055063 ↗
3. Deep learning based synthesis of MRI, CT and PET: Review and analysis10.1016/j.media.2023.103046 ↗
4. From CNNs to GANs for cross-modality medical image estimation10.1016/j.compbiomed.2022.105556 ↗
5. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-Net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.10.1007/978-3-319-24574-4_28 ↗
6. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., and Houlsby, N. (2021, January 3–7). An image is worth 16×16 words: Transformers for image recognition at scale. Proceedings of the International Conference on Learning Representations (ICLR), Virtual.
7. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., and Bengio, Y. (2014, January 8–13). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
8. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
9. A survey on vision transformer10.1109/TPAMI.2022.3152247 ↗
10. Transformers in vision: A survey10.1145/3505244 ↗
11. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., and Guo, B. (2021, January 10–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00986 ↗
12. Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., and Wang, M. (2022, January 23–27). Swin-Unet: Unet-like pure transformer for medical image segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Tel Aviv, Israel.10.1007/978-3-031-25066-8_9 ↗
13. Random forest regression for magnetic resonance image synthesis10.1016/j.media.2016.08.009 ↗
14. Magnetic resonance image example-based contrast synthesis10.1109/TMI.2013.2282126 ↗
15. Cross-modality image synthesis via weakly coupled and geometry co-regularized joint dictionary learning10.1109/TMI.2017.2781192 ↗
16. Deep learning-based convolutional neural network for intramodality brain MRI synthesis10.1002/acm2.13530 ↗
17. Synthesis of magnetic resonance images from computed tomography data using convolutional neural network with contextual loss function10.21037/qims-21-846 ↗
18. Isola, P., Zhu, J.-Y., Zhou, T., and Efros, A.A. (2017, January 21–26). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.632 ↗
19. Nie, D., Trullo, R., Lian, J., Petitjean, C., Ruan, S., Wang, Q., and Shen, D. (2017, January 10–14). Medical image synthesis with context-aware generative adversarial networks. Proceedings of the Medical Image Computing and Computer-Assisted Intervention (MICCAI), Quebec, QC, Canada.10.1007/978-3-319-66179-7_48 ↗
20. Image synthesis in multi-contrast MRI with conditional generative adversarial networks10.1109/TMI.2019.2901750 ↗
21. Missing MRI pulse sequence synthesis using multi-modal generative adversarial network10.1109/TMI.2019.2945521 ↗
22. T1-weighted and T2-weighted MRI image synthesis with convolutional generative adversarial networks10.5603/RPOR.a2021.0005 ↗
23. Bi-MGAN: Bidirectional T1-to-T2 MRI images prediction using multi-generative multi-adversarial nets10.1016/j.bspc.2022.103994 ↗
24. Modal disentangled generative adversarial networks for bidirectional magnetic resonance image synthesis10.1016/j.engappai.2024.109817 ↗
25. Zhu, J.-Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22–29). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.244 ↗
26. Yang, H., Sun, J., Carass, A., Zhao, C., Lee, J., Xu, Z., and Prince, J. (2018, January 20). Unpaired brain MR-to-CT synthesis using a structure-constrained CycleGAN. Proceedings of the Deep Learning in Medical Image Analysis, Granada, Spain.10.1007/978-3-030-00889-5_20 ↗
27. Mao, X., Li, Q., Xie, H., Lau, R.Y.K., Wang, Z., and Smolley, S.P. (2017, January 22–29). Least squares generative adversarial networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.304 ↗
28. Skandarani, Y., Jodoin, P.-M., and Lalande, A. (2023). GANs for medical image synthesis: An empirical study. J. Imaging, 9.10.3390/jimaging9030069 ↗
29. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., and Zhou, Y. (2021). TransUNet: Transformers make strong encoders for medical image segmentation. arXiv.
30. ResViT: Residual vision transformers for multimodal medical image synthesis10.1109/TMI.2022.3167808 ↗
31. Wu, Y., and He, K. (2018, January 8–14). Group normalization. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01261-8_1 ↗
32. Ramachandran, P., Zoph, B., and Le, Q.V. (2017). Searching for activation functions. arXiv.
33. Wang, H., Zhu, Y., Green, B., Adam, H., Yuille, A., and Chen, L.-C. (2020, January 23–28). Axial-DeepLab: Stand-alone axial-attention for panoptic segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Virtual.10.1007/978-3-030-58548-8_7 ↗
34. Xiong, R., Yang, Y., He, D., Zheng, K., Zheng, S., Xing, C., Zhang, H., Lan, Y., Wang, L., and Liu, T. (2020, January 13–18). On layer normalization in the transformer architecture. Proceedings of the International Conference on Machine Learning (ICML), Virtual.
35. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
36. Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2016). Instance normalization: The missing ingredient for fast stylization. arXiv.
37. Loss functions for image restoration with neural networks10.1109/TCI.2016.2644865 ↗
38. Johnson, J., Alahi, A., and Fei-Fei, L. (2016, January 11–14). Perceptual losses for real-time style transfer and super-resolution. Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands.10.1007/978-3-319-46475-6_43 ↗
39. Image quality assessment: From error visibility to structural similarity10.1109/TIP.2003.819861 ↗
40. SimpleITK image-analysis notebooks: A collaborative environment for education and reproducible research10.1007/s10278-017-0037-8 ↗