journal article·2025-12-15Open access

Deep Learning-Based Fusion of Multimodal MRI Features for Brain Tumor Detection

Bakhita Salman, Eithar Yassin, Deepak Ganta, Hermes Luna

Applied Sciences· Volume 15· Issue 24

Abstract

Despite advances in deep learning, brain tumor detection from MRI continues to face major challenges, including the limited robustness of single-modality models, the computational burden of transformer-based architectures, opaque fusion strategies, and the lack of efficient binary screening tools. To address these issues, we propose a lightweight multimodal CNN framework that integrates T1, T2, and FLAIR MRI sequences using modality-specific encoders and a channel-wise fusion module (concatenation followed by a 1 × 1 convolution). The pipeline incorporates U-Net-based segmentation for tumor-focused patch extraction, improving localization and reducing irrelevant background. Evaluated on the BraTS 2020 dataset (7500 slices; 70/15/15 patient-level split), the proposed model achieves 93.8% accuracy, 94.1% F1-score, and 19 ms inference time. It outperforms all single-modality ablations by up to 5% and achieves competitive or superior performance to transformer-based baselines while using over 98% fewer parameters. Grad-CAM and LIME visualizations further confirm clinically meaningful tumor-region activation. Overall, this efficient and interpretable multimodal framework advances scalable brain tumor screening and supports integration into real-time clinical workflows.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Brain Tumor Detection and Classification · Advanced Neural Network Applications · Glioma Diagnosis and Treatment

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Engineering, Texas A&M International University, Laredo, TX 78041, USA

University of Medical Sciences and Technology, Khartoum P.O. Box 12810, Sudan

Funding

TAMIU University research grant

References (34)

Show source-provided references
1. Menze, B., and Bauer, A. (2025, September 10). Brain Tumor Segmentation (BraTS2020): Benchmarks and Dataset. Kaggle Datasets. June 2020. Available online: https://www.kaggle.com/datasets/awsaf49/brats2020-training-data.
2. Taher, F., Shoaib, M.R., Emara, H.M., Abdelwahab, K.M., El-Samie, F.E.A., and Haweel, M.T. (2022). Efficient framework for brain tumor detection using different deep learning techniques. PLoS ONE, 10.10.3389/fpubh.2022.959667 ↗
3. Dhar, J., Zaidi, N., Haghighat, M., Roy, S., Goyal, P., Alavi, A., and Kumar, V. (March, January 26). Multimodal Fusion Learning with Dual Attention for Medical Imaging. Proceedings of the 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Tucson, AZ, USA. Available online: https://openaccess.thecvf.com/content/WACV2025/papers/Dhar_Multimodal_Fusion_Learning_with_Dual_Attention_for_Medical_Imaging_WACV_2025_paper.pdf.10.1109/WACV61041.2025.00428 ↗
4. Guo, S., Wang, L., Chen, Q., Wang, L., Zhang, J., and Zhu, Y. (2022). Multimodal MRI Image Decision Fusion-Based Network for Glioma Classification. Front. Neurosci., 12.10.3389/fonc.2022.819673 ↗
5. Kumar, V., Joshi, K., Kanti, P., Reshi, J.S., Rawat, G., and Kumar, A. (2023, January 23–25). Brain Tumor Diagnosis using Image Fusion and Deep Learning. Proceedings of the 2023 IEEE International Conference on Advances in Computing and Communication Engineering (ICACCE), Erode, India. Available online: https://ieeexplore.ieee.org/document/10104937.10.1109/ICSCDS56580.2023.10104937 ↗
6. A Novel Multimodal Medical Image Fusion Framework with Edge Preservation and Transformer10.1038/s41598-025-93616-y ↗
7. Polinati, S., Bavirisetti, D.P., Rajesh, K.N.V.P.S., Naik, G.R., and Dhuli, R. (2021). The Fusion of MRI and CT Medical Images Using Variational Mode Decomposition. Appl. Sci., 11.10.3390/app112210975 ↗
8. A review of cancer data fusion methods based on deep learning10.1016/j.inffus.2024.102361 ↗
9. Optimized deep learning for brain tumor detection: A hybrid approach with attention mechanisms and clinical explainability10.1038/s41598-025-04591-3 ↗
10. ResSAXU-Net for multimodal brain tumor segmentation from brain MRI
11. Brain tumor classification using deep CNN features via transfer learning10.1016/j.compbiomed.2019.103345 ↗
12. Brain Tumor Segmentation for Multi-Modal MRI with Missing Information10.1007/s10278-023-00860-7 ↗
13. Gradient-based multiresolution image fusion10.1109/TIP.2004.823821 ↗
14. Huang, J., Yagmurlu, B., Molleti, P., Lee, R., VanderPloeg, A., Noor, H., Bareja, R., Li, Y., Iv, M., and Itakura, H. (2025). Brain tumor segmentation using deep learning: High performance with minimized MRI data. Front. Radiol., 3, Available online: https://www.frontiersin.org/journals/radiology/articles/10.3389/fradi.2025.1616293/full.10.3389/fradi.2025.1616293 ↗
15. Deep CNN-Based Multi-Grade Brain Tumor Classification with Enhanced Data Augmentation
16. Combined spiral transformation and model-driven multi-modal deep learning scheme for automatic prediction of TP53 mutation in pancreatic cancer10.1109/TMI.2020.3035789 ↗
17. Srinivasan, S., Francis, D., Mathivanan, S.K., Rajadurai, H., Shivahare, B.D., and Shah, M.A. (2024). A hybrid deep CNN model for brain tumor image multi-classification. BMC Med. Imaging, 24, Available online: https://www.researchgate.net/publication/377532396_A_hybrid_deep_CNN_model_for_brain_tumor_image_multi-classification.10.1186/s12880-024-01195-7 ↗
18. Song, J., Zheng, J., Li, P., Lu, X., Zhu, G., and Shen, P. (2021). An effective multimodal image fusion method using MRI and PET for Alzheimer’s disease diagnosis. Front. Digit. Health, 3.10.3389/fdgth.2021.637386 ↗
19. Multi channel fusion diffusion models for brain tumor MRI data augmentation10.1038/s41598-025-06529-1 ↗
20. Luo, T., and Xu, W. (2025). MMIF-AMIN: Adaptive Loss-Driven Multi-Scale Invertible Dense Network for Multimodal Medical Image Fusion. arXiv, Available online: https://arxiv.org/html/2508.08679v1.
21. Wavelet Attention is all you need in multimodal medical image fusion10.1016/j.neucom.2025.131448 ↗
22. Multiple Attention Channels Aggregated Network for Multimodal Medical Image Fusion10.1002/mp.17607 ↗
23. Dharshini, S., Geetha, S., Arya, S., Mekala, N., Reshma, R., and Sasirekha, S.P. (2023, January 2–4). An Enhanced Brain Tumor Detection Scheme using a Hybrid Deep Learning Model. Proceedings of the 2023 Second International Conference on Electronics and Renewable Systems (ICEARS), Tuticorin, India. Available online: https://ieeexplore.ieee.org/document/10085267/.10.1109/ICEARS56392.2023.10085267 ↗
24. Fusion of MRI and CT images for brain tumor segmentation and classification
25. Deep fusion of multi-modal features for brain tumor image segmentation10.1016/j.heliyon.2023.e19266 ↗
26. A deep learning approach for brain tumor detection from MRI images
27. Enhanced Brain Tumor Segmentation Using CBAM-Integrated U-Net10.1155/ijbi/2149042 ↗
28. Brain tumor segmentation with deep learning: Current approaches and future trends
29. Advanced techniques for brain MRI processing and segmentation: A survey
30. A Survey on Advanced Segmentation Techniques for Brain MRI Image Segmentation
31. Enhanced MRI brain tumor detection using deep learning in conjunction with explainable AI SHAP based diverse and multi feature analysis
32. The role of generative adversarial networks in brain MRI: A scoping review
33. Filho, L.M.S., Santos, T.M.A., and Oliveira, M.S. (2018, January 11–13). A review of MRI segmentation techniques. Proceedings of the IEEE International Symposium on Biomedical Imaging, Barcelona, Spain. Available online: https://www.researchgate.net/publication/281772814_A_REVIEW_ON_MRI_IMAGE_SEGMENTATION_TECHNIQUES.
34. Advanced convolutional neural network with attention mechanism for Alzheimer’s disease classification using MRI