journal article·2022-11-29Open access

Two-View Mammogram Synthesis from Single-View Data Using Generative Adversarial Networks

Asumi Yamazaki, Takayuki Ishida

Applied Sciences· Volume 12· Issue 23

Abstract

While two-view mammography taking both mediolateral-oblique (MLO) and cranio-caudual (CC) views is the current standard method of examination in breast cancer screening, single-view mammography is still being performed in some countries on women of specific ages. The rate of cancer detection is lower with single-view mammography than for two-view mammography, due to the lack of available image information. The goal of this work is to improve single-view mammography’s ability to detect breast cancer by providing two-view mammograms from single projections. The synthesis of novel-view images from single-view data has recently been achieved using generative adversarial networks (GANs). Here, we apply complete representation GAN (CR-GAN), a novel-view image synthesis model, aiming to produce CC-view mammograms from MLO views. Additionally, we incorporate two adaptations—the progressive growing (PG) technique and feature matching loss—into CR-GAN. Our results show that use of the PG technique reduces the training time, while the synthesized image quality is improved when using feature matching loss, compared with the method using only CR-GAN. Using the proposed method with the two adaptations, CC views similar to real views are successfully synthesized for some cases, but not all cases; in particular, image synthesis is rarely successful when calcifications are present. Even though the image resolution and quality are still far from clinically acceptable levels, our findings establish a foundation for further improvements in clinical applications. As the first report applying novel-view synthesis in medical imaging, this work contributes by offering a methodology for two-view mammogram synthesis.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Generative Adversarial Networks and Image Synthesis · AI in cancer detection · Advanced Image Processing Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Division of Health Sciences, Osaka University Graduate School of Medicine, Suita 565-0871, Japan

References (52)

Show source-provided references
1. Terrasse, V. (2020). Latest global cancer data: Cancer burden rises to 19.3 million new cases and 10.0 million cancer deaths in 2020. The International Agency for Research on Cancer Press Release 292, IARC.
2. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries10.3322/caac.21660 ↗
3. Survival of women treated for early breast cancer detected by screening is same as in general population, audit shows
4. Effect of mammographic screening from age 40 years on breast cancer mortality (UK Age trial): Final results of a randomised, controlled trial10.1016/S1470-2045(20)30398-3 ↗
5. A meta-analysis of mammographic screening with and without clinical breast examination10.1111/cas.12693 ↗
6. Benefits and harms of breast cancer screening with mammography in women aged 40–49 years: A systematic review10.1002/ijc.30794 ↗
7. Change in effectiveness of mammography screening with decreasing breast cancer mortality: A population-based study10.1093/eurpub/ckac047 ↗
8. Digital mammography in young women: Is a single view sufficient?
9. Rubin, D. (2019). Guidance on Screening and Symptomatic Breast Imaging, The Royal College of Radiologists. [4th ed.].
10. Baseline screening mammography: One vs two views per breast10.2214/ajr.147.6.1149 ↗
11. Screening mammography: A successful public health initiative10.1590/S1020-49892006000800009 ↗
12. Screening mammographyin women 40-49 years old: Current evidence10.2214/AJR.17.18707 ↗
13. Problems in breast cancer screening
14. At what age should screening mammography be recommended for Asian women?10.1002/cam4.468 ↗
15. Screening mammography in women aged 40–49: Is it time to change?10.1186/1477-7800-3-4 ↗
16. Interpreting one-view mammographic findings: Minimizing callbacks while maximizing cancer detection10.1148/rg.344130066 ↗
17. A survey on deep learning in medical image analysis10.1016/j.media.2017.07.005 ↗
18. Bakator, M., and Radosav, D. (2018). Deep learning and medical diagnosis: A review of literature. Multimodal Technol. Interact., 2.10.3390/mti2030047 ↗
19. Deep learning in medical imaging and radiation therapy10.1002/mp.13264 ↗
20. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. Advances in Neural Information Processing Systems (NIPS 2014), NeurIPS.
21. Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv.
22. Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6–11). Wasserstein generative adversarial networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia.
23. Karras, T., Aila, T., Laine, S., and Lehtinen, J. (May, January 30). Progressive growing of gans for improved quality, stability, and variation. Proceedings of the Sixth International Conference on Learning Representations (ICLR), Vancouver, BC, Canada.
24. Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., and Aila, T. (2020, January 13–19). Analyzing and improving the image quality of StyleGAN. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00813 ↗
25. Identifying women with mammographically- occult breast cancer leveraging GAN- simulated mammograms10.1109/TMI.2021.3108949 ↗
26. Korkinof, D., Rijken, T., O’Neill, M., Yearsley, J., Harvey, H., and Glocker, B. (2018). High-resolution mammogram synthesis using progressive generative adversarial networks. arXiv.
27. A generative adversarial network for synthetization of regions of interest based on digital mammograms10.1038/s41598-022-09929-9 ↗
28. Tran, L., Yin, X., and Liu, X. (2017, January 21–26). Disentangled representation learning GAN for pose-invariant face recognition. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.141 ↗
29. Zhao, B., Wu, X., Cheng, Z., Liu, H., Jie, Z., and Feng, J. (2018). Multi-view image generation from a single-view. arXiv.10.1145/3240508.3240536 ↗
30. Frontal face generation algorithm from multi-view images based on generative adversarial network10.33851/JMIS.2021.8.2.85 ↗
31. Zou, H., Ak, K.E., and Kassim, A.A. (2020, January 25–28). Edge-Gan: Edge conditioned multi-view face image generation. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.10.1109/ICIP40778.2020.9190723 ↗
32. Tian, Y., Peng, X., Zhao, L., Zhang, S., and Metaxas, D.N. (2018). CR-GAN: Learning complete representations for multi-view generation. arXiv.10.24963/ijcai.2018/131 ↗
33. Jahanian, A., Puig, X., Tian, Y., and Isola, P. (2021). Generative models as a data source for multiview representation learning. arXiv.
34. (2022, October 31). bluer555/CR-GAN. Available online: https://github.com/bluer555/CR-GAN.
35. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A. (2017). Improved training of Wasserstein GANs. NeurIPS Proceedings, NeurIPS.
36. Wang, T., Liu, M., Zhu, J., Tao, A., Kautz, J., and Catanzaro, B. (2018, January 18–23). High-resolution image synthesis and semantic manipulation with conditional gans. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00917 ↗
37. Johnson, J., Alahi, A., and Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. Lecture Notes in Computer Science, Springer.10.1007/978-3-319-46475-6_43 ↗
38. Donahue, J., Krähenbühl, P., and Darrell, T. (2016). Adversarial feature learning. arXiv.
39. A curated mammography data set for use in computer-aided detection and diagnosis research10.1038/sdata.2017.177 ↗
40. INbreast: Toward a full-field digital mammographic database10.1016/j.acra.2011.09.014 ↗
41. (2022, October 31). The Chinese Mammography Database (CMMD). Available online: https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=70230508.
42. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. (2017). GANs trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems (NIPS 2017), NeurIPS.
43. Borji, A. (2018). Pros and cons of GAN evaluation measures. arXiv.10.1016/j.cviu.2018.10.009 ↗
44. Image quality assessment: From error measurement to structural similarity10.1109/TIP.2003.819861 ↗
45. Wang, Z., Simoncelli, E.P., and Bovik, A.C. (2003, January 9–12). Multiscale structural similarity for image quality assessment. Proceedings of the 37th IEEE Asilomar Conference on Signals, Systems & Computers, Pacific Grove, CA, USA.
46. Jaiswal, A., Babu, A.R., Zadeh, M.Z., and Banerjee, D. (2021). A survey on contrastive self-supervised learning. arXiv.10.3390/technologies9010002 ↗
47. Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020). A simple framework for contrastive learning of visual representations. arXiv.
49. Liu, Z., Luo, P., Wang, X., and Tang, X. (2015, January 7–13). Deep learning face attributes in the wild. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.425 ↗
50. Image quality assessment through FSIM, SSIM, MSE and PSNR—A comparative study10.4236/jcc.2019.73002 ↗
51. Pambrun, J.-F., and Noumeir, R. (2015, January 27–30). Limitations of the SSIM quality metric in the context of diagnostic imaging. Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada.10.1109/ICIP.2015.7351345 ↗
52. Mudeng, V., Kim, M., and Choe, S. (2022). Prospects of structual similarity index for medical image analysis. Appl. Sci., 12.10.3390/app12083754 ↗