Automated Chest X-Ray Diagnosis Report Generation with Cross-Attention Mechanism
Source abstract
Abstract
In the medical field, it is extremely important to use deep learning technology to automatically generate diagnostic reports for chest X-ray images. This technology provides an effective solution to the challenges faced by the medical field in processing large numbers of chest X-ray images. Especially during large-scale outbreaks of epidemics such as the new COVID-19, rapid and accurate screening and diagnosis of cases become important tasks. This study uses deep learning technology to automatically generate diagnostic reports for chest X-ray images, which significantly reduces the workload of doctors, reduces the risk of misdiagnosis and missed diagnosis, and provides technical support for improving public health emergency response capabilities. In this study, we propose an innovative network architecture to address the limitations of traditional image description networks in generating chest X-ray diagnostic reports, especially the large area deviation between abnormal and normal areas, and the lack of effective alignment of the two modalities of image and text. The convolutional block attention module (CBAM) is adopted to effectively alleviate the data bias problem through a sophisticated feature attention mechanism and improve the model’s ability to recognize abnormal image areas. The cross-attention mechanism is adopted to optimize the alignment process between images and texts, ensuring the accuracy and reliability of the diagnosis report.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Radiomics and Machine Learning in Medical Imaging · COVID-19 diagnosis using AI
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science and Technology, Changchun University, Changchun 130022, China
Jilin Provincial Key Laboratory of Human Health Status Identification Function & Enhancement, Changchun 130022, China
Key Laboratory of Intelligent Rehabilitation and Barrier-Free for the Disabled, Changchun University, Ministry of Education, Changchun 130022, China
College of Electronic Information Engineering, Changchun University, Changchun 130012, China
Funding
Education Department of Jilin Province
References (25)
Show source-provided references
1. Evidence-based guideline for the written radiology report: Methods, recommendations and implementation challenges10.1111/1754-9485.12014 ↗
2. Measuring and managing radiologist workload: Measuring radiologist reporting times using data from a Radiology Information System10.1111/1754-9485.12092 ↗
3. Radiologist burnout: Trends in medical imaging utilization under the national health insurance system with the universal code bundling strategy in an academic tertiary medical centre10.1016/j.ejrad.2022.110596 ↗
4. A systematic review of fatigue in radiology: Is it a problem?10.2214/AJR.17.18613 ↗
5. Xue, Y., Xu, T., Rodney Long, L., Xue, Z., Antani, S., Thoma, G.R., and Huang, X. (2018, January 16–20). Multimodal recurrent model with attention for automated radiology report generation. Proceedings of the Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain. Proceedings, Part I.10.1007/978-3-030-00928-1_52 ↗
6. Thawkar, O., Shaker, A., Mullappilly, S.S., Cholakkal, H., Anwer, R.M., Khan, S., Laaksonen, J., and Khan, F.S. (2023). Xraygpt: Chest radiographs summarization using medical vision-language models. arXiv.10.18653/v1/2024.bionlp-1.35 ↗
7. Radiology report generation with a learned knowledge base and multi-modal alignment10.1016/j.media.2023.102798 ↗
8. Li, C.Y., Liang, X., Hu, Z., and Xing, E.P. (February, January 27). Knowledge-driven encode, retrieve, paraphrase for medical image report generation. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
9. Chen, Z., Shen, Y., Song, Y., and Wan, X. (2022). Cross-modal memory networks for radiology report generation. arXiv.10.18653/v1/2021.acl-long.459 ↗
10. Zhang, Y., Jiang, H., Miura, Y., Manning, C.D., and Langlotz, C.P. (2022, January 5–6). Contrastive learning of medical visual representations from paired images and text. Proceedings of the Machine Learning for Healthcare Conference, PMLR, Durham, NC, USA.
11. Wang, Z., Wu, Z., Agarwal, D., and Sun, J. (2022). Medclip: Contrastive learning from unpaired medical images and text. arXiv.10.18653/v1/2022.emnlp-main.256 ↗
12. Wang, X., Peng, Y., Lu, L., Lu, Z., and Summers, R.M. (2018, January 18–23). Tienet: Text-image embedding network for common thorax disease classification and reporting in chest x-rays. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00943 ↗
13. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European conference on computer vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
14. Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems.
15. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
16. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
17. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, National Science Foundation.
18. Girshick, R. (2015, January 7–13). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
19. Flandoli, F., Morlacchi, S., and Papini, A. (2022, January 26–29). Effect of Transport Noise on Kelvin–Helmholtz Instability. Proceedings of the Stochastic Transport in Upper Ocean Dynamics Annual Workshop, London, UK.10.1007/978-3-031-40094-0_3 ↗
20. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (July, January 26). Learning deep features for discriminative localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
21. Grad-CAM: Visual explanations from deep networks via gradient-based localization10.1007/s11263-019-01228-7 ↗
22. Kingma, D.P. (2014). Adam: A method for stochastic optimization. arXiv.
23. Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 6–12). Bleu: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.10.3115/1073083.1073135 ↗
24. Lin, C.Y., and Hovy, E. (June, January 27). Automatic evaluation of summaries using n-gram co-occurrence statistics. Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, Edmonton, AB, Canada.10.3115/1073445.1073465 ↗
25. Banerjee, S., and Lavie, A. (2005, January 29). METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Ann Arbor, MI, USA.