journal article·2025-04Open access

Vision GNN (ViG) architecture for a fine-tuned segmentation of a complex Al–Si metal matrix composite XCT volume

M. Lapenna, A. Tsamos, F. Faglioni, R. Fioresi, F. Zanchetta, G. Bruno

Journal of Materials Science· Volume 60· Issue 16

Abstract

Abstract In this paper, we implement a vision graph neural network (ViG) architecture to segment microstructures in X-ray computed tomography 3D data. Our ViG architecture is first trained on a synthetic augmented dataset, and then fine-tuned on experimental data to obtain an improved segmentation. Successively, we assess the accuracy of the segmentation on manually-labeled experimental slices. We exemplarily use the approach on a complex microstructure: a metal matrix composite, reinforced with two ceramic phases, intermetallic inclusions and a silicon network, in order to show the generality of our method. ViG model proves to be more efficient than U-Nets in adapting to new data when fine-tuned on a small portion of the experimental data. The fine-tuned ViG shows comparable performance to U-Nets, while largely reducing the number of trainable parameters, with the potential of greater adaptability and efficiency.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Industrial Vision Systems and Defect Detection · Advanced Surface Polishing Techniques · Electron and X-Ray Spectroscopy Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (41)

Show source-provided references
1. Choudhary K et al (2021) Recent advances and applications of deep learning methods in materials science. npj Comput Mater 8:1–26
2. Strohmann T et al (2019) Semantic segmentation of synchrotron tomography of multiphase al-si alloys using a convolutional neural network with a pixel-wise weighted loss function. Sci Rep 1–1010.1038/s41598-019-56008-7 ↗
3. Evsevleev S, Paciornik S, Bruno G (2020) Advanced deep learning-based 3d microstructural characterization of multiphase metal matrix composites. Adv Eng Mater 22:1–610.1002/adem.201901197 ↗
4. Tsamos A, Evsevleev S, Fioresi R, Faglioni F, Bruno G (2023) Synthetic data generation for automatic segmentation of x-ray computed tomography reconstructions of complex microstructures. J Imaging 9:1–2310.3390/jimaging9020022 ↗
5. Wong VWH, Ferguson M, Law KH, Lee YTT, Witherell P (2021) Automatic volumetric segmentation of additive manufacturing defects with 3d u-net. arXiv:2101.0899310.1115/DETC2021-68885 ↗
6. Du W et al (2020) Automated detection of defects with low semantic information in x-ray images based on deep learning. J Intell Manuf 32:141–15610.1007/s10845-020-01566-1 ↗
7. Fotos G, Campbell A, Murray P, Yakushina E (2023) Deep learning enhanced watershed for microstructural analysis using a boundary class semantic segmentation. J Mater Sci 58:1–2110.1007/s10853-023-08901-w ↗
8. Sarvamangala DR, Kulkarni RV (2021) Convolutional neural networks in medical image understanding: a survey. Evol Intell 15:1–2210.1007/s12065-020-00540-3 ↗
9. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86:2278–232410.1109/5.726791 ↗
10. Krizhevsky A, Sutskever I, Hinton GE, Pereira F, Burges C, Bottou L, Weinberger K (eds) (2012) Imagenet classification with deep convolutional neural networks. (eds Pereira, F., Burges, C., Bottou, L. & Weinberger, K.) Advances in Neural Information Processing Systems, Vol. 25 (Curran Associates, Inc., 2012), pp 1–9
11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. 2016 IEEE conference on computer vision and pattern recognition (CVPR) 770–77810.1109/CVPR.2016.90 ↗
12. Carion N et al (2020) End-to-end object detection with transformers. Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I 213-229 . https://doi.org/10.1007/978-3-030-58452-8_1310.1007/978-3-030-58452-8_13 ↗
13. Dosovitskiy A et al (2020) An image is worth 16x16 words: Transformers for image recognition at scale. pp 1–22. arXiv:2010.11929. https://api.semanticscholar.org/CorpusID:225039882
14. Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P (2017) Geometric deep learning: going beyond euclidean data. IEEE Signal Proc Mag 34:18–4210.1109/MSP.2017.2693418 ↗
15. Bronstein MM, Bruna J, Cohen T, Velivckovic P (2021) Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. pp 1–160. arXiv:abs/2104.13478. https://api.semanticscholar.org/CorpusID:233423603
16. Landrieu L, Simonovsky M (2017) Large-scale point cloud semantic segmentation with superpoint graphs. 2018 IEEE/CVF conference on computer vision and pattern recognition 4558–4567. https://api.semanticscholar.org/CorpusID:439683710.1109/CVPR.2018.00479 ↗
17. Wang Y et al (2019) Dynamic graph cnn for learning on point clouds. ACM Trans Graph 38:1–1310.1145/3355089.3356541 ↗
18. Yang Y, Qiu J, Song M, Tao D, Wang X (2020) Distilling knowledge from graph convolutional networks. 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR) 7072–7081. https://api.semanticscholar.org/CorpusID:21471384810.1109/CVPR42600.2020.00710 ↗
19. Rusch TK, Bronstein MM, Mishra S (2023) A survey on oversmoothing in graph neural networks. pp 1–10. arXiv:abs/2303.10993. https://api.semanticscholar.org/CorpusID:257632346
20. Han K, Wang Y, Guo J, Tang Y, Wu E (2022) Vision gnn: an image is worth graph of nodes. In: Proceedings of the 36th international conference on neural information processing systems. pp 1–15
21. Lapenna M et al (2024) Geometric deep learning for enhanced quantitative analysis of microstructures in x-ray computed tomography data. Discover Appl Sci 6:1–910.1007/s42452-024-05985-0 ↗
22. Kipf T, Welling M (2016) Semi-supervised classification with graph convolutional networks. pp 1–14. arXiv:abs/1609.02907. https://api.semanticscholar.org/CorpusID:31442186
23. Zachary WW (1977) An information flow model for conflict and fission in small groups. J Anthropol Res 33:452–47310.1086/jar.33.4.3629752 ↗
24. Buades A, Coll B, Morel J.-M (2005) A non-local algorithm for image denoising. In: Computer society conference on computer vision and pattern recognition, vol 2. pp 60–6510.1109/CVPR.2005.38 ↗
25. Tsamos A, Evsevleev S, Bruno G (2023) Noise and blur removal from corrupted x-ray computed tomography scans: A multilevel and multiscale deep convolutional framework approach with synthetic training data (bam synthcond). Tomography Mater Struct 1–1610.1016/j.tmater.2023.100011 ↗
26. Tsamos A, Evsevleev S, Fioresi R, Faglioni F, Bruno G (2023) A complete strategy to achieve high precision automatic segmentation of challenging experimental x-ray computed tomography data using low-resemblance synthetic training data. Adv Eng Mater 26:1–910.1002/adem.202301030 ↗
27. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, pp 330–371. http://www.deeplearningbook.org
28. Velickovic P (2022) Message passing all the way up. pp 1–10. arXiv:abs/2202.11097. https://api.semanticscholar.org/CorpusID:247026068
29. Veličković P et al (2018) Graph attention networks. In: International conference on learning representations. pp 1–12
30. Keyulu Xu J. L, Weihua Hu, Jegelka S (2019) How powerful are graph neural networks? In: International conference on learning representations. pp 1–17. https://openreview.net/forum?id=ryGs6iA5Km
31. Weisfeiler B, Leman A (1968) The reduction of a graph to a canonical form and the algebra arising during this reduction. Nauchno-Technicheskaya Informatsia 9:1–11
32. Shervashidze Nino VLEJ, Schweitzer P, Kurt M, Borgwardt KM (2011) Weisfeiler-lehman graph kernels. J Mach Learn Res 12:1–23
33. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st international conference on neural information processing systems. pp 1025–1035
34. He P, Qu A, Xiao S, Ding M (2023) A gnn-based network for tissue semantic segmentation in histopathology image. J Phys: Conf Ser 2504:012047
35. Stefan Elfwing EU, Doya K (2017) Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. pp 1–18. arxiv.org/abs/1702.03118
36. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: 3rd International conference on learning representations. ICLR, pp 1–15
37. Genç A, Kovarik L, Fraser HL (2022) A deep learning approach for semantic segmentation of unbalanced data in electron tomography of catalytic materials. Sci Rep 12:1–1210.1038/s41598-022-16429-3 ↗
38. Thambawita V et al (2022) Singan-seg: Synthetic training data generation for medical image segmentation. PLOS ONE 17:1–2410.1371/journal.pone.0267976 ↗
39. Nguyen Q, Vu T, Tran A, Nguyen K (2023) Diffusion-based synthetic data generation for pixel-level semantic segmentation. Adv Neural Inf Proc Syst 36:1–21
40. Corvi R, Cozzolino D, Poggi G, Nagano K, Verdoliva L (2023) Intriguing properties of synthetic images: from generative adversarial networks to diffusion models. In: 2023 IEEE/CVF Conference on computer vision and pattern recognition workshops (CVPRW). pp 973–982. https://api.semanticscholar.org/CorpusID:25810786910.1109/CVPRW59228.2023.00104 ↗
41. Gao H, Ji S (2022) Graph u-nets. IEEE Trans Pattern Anal Mach Intell 44:4948–4960