Vision GNN (ViG) architecture for a fine-tuned segmentation of a complex Al–Si metal matrix composite XCT volume
Source abstract
Abstract
Abstract In this paper, we implement a vision graph neural network (ViG) architecture to segment microstructures in X-ray computed tomography 3D data. Our ViG architecture is first trained on a synthetic augmented dataset, and then fine-tuned on experimental data to obtain an improved segmentation. Successively, we assess the accuracy of the segmentation on manually-labeled experimental slices. We exemplarily use the approach on a complex microstructure: a metal matrix composite, reinforced with two ceramic phases, intermetallic inclusions and a silicon network, in order to show the generality of our method. ViG model proves to be more efficient than U-Nets in adapting to new data when fine-tuned on a small portion of the experimental data. The fine-tuned ViG shows comparable performance to U-Nets, while largely reducing the number of trainable parameters, with the potential of greater adaptability and efficiency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Advanced Surface Polishing Techniques · Electron and X-Ray Spectroscopy Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (41)
Show source-provided references
1. Choudhary K et al (2021) Recent advances and applications of deep learning methods in materials science. npj Comput Mater 8:1–26
2. Strohmann T et al (2019) Semantic segmentation of synchrotron tomography of multiphase al-si alloys using a convolutional neural network with a pixel-wise weighted loss function. Sci Rep 1–1010.1038/s41598-019-56008-7 ↗
3. Evsevleev S, Paciornik S, Bruno G (2020) Advanced deep learning-based 3d microstructural characterization of multiphase metal matrix composites. Adv Eng Mater 22:1–610.1002/adem.201901197 ↗
4. Tsamos A, Evsevleev S, Fioresi R, Faglioni F, Bruno G (2023) Synthetic data generation for automatic segmentation of x-ray computed tomography reconstructions of complex microstructures. J Imaging 9:1–2310.3390/jimaging9020022 ↗
5. Wong VWH, Ferguson M, Law KH, Lee YTT, Witherell P (2021) Automatic volumetric segmentation of additive manufacturing defects with 3d u-net. arXiv:2101.0899310.1115/DETC2021-68885 ↗
6. Du W et al (2020) Automated detection of defects with low semantic information in x-ray images based on deep learning. J Intell Manuf 32:141–15610.1007/s10845-020-01566-1 ↗
7. Fotos G, Campbell A, Murray P, Yakushina E (2023) Deep learning enhanced watershed for microstructural analysis using a boundary class semantic segmentation. J Mater Sci 58:1–2110.1007/s10853-023-08901-w ↗
8. Sarvamangala DR, Kulkarni RV (2021) Convolutional neural networks in medical image understanding: a survey. Evol Intell 15:1–2210.1007/s12065-020-00540-3 ↗
9. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86:2278–232410.1109/5.726791 ↗
10. Krizhevsky A, Sutskever I, Hinton GE, Pereira F, Burges C, Bottou L, Weinberger K (eds) (2012) Imagenet classification with deep convolutional neural networks. (eds Pereira, F., Burges, C., Bottou, L. & Weinberger, K.) Advances in Neural Information Processing Systems, Vol. 25 (Curran Associates, Inc., 2012), pp 1–9
11. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. 2016 IEEE conference on computer vision and pattern recognition (CVPR) 770–77810.1109/CVPR.2016.90 ↗
12. Carion N et al (2020) End-to-end object detection with transformers. Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I 213-229 . https://doi.org/10.1007/978-3-030-58452-8_1310.1007/978-3-030-58452-8_13 ↗
13. Dosovitskiy A et al (2020) An image is worth 16x16 words: Transformers for image recognition at scale. pp 1–22. arXiv:2010.11929. https://api.semanticscholar.org/CorpusID:225039882
14. Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P (2017) Geometric deep learning: going beyond euclidean data. IEEE Signal Proc Mag 34:18–4210.1109/MSP.2017.2693418 ↗
15. Bronstein MM, Bruna J, Cohen T, Velivckovic P (2021) Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. pp 1–160. arXiv:abs/2104.13478. https://api.semanticscholar.org/CorpusID:233423603
16. Landrieu L, Simonovsky M (2017) Large-scale point cloud semantic segmentation with superpoint graphs. 2018 IEEE/CVF conference on computer vision and pattern recognition 4558–4567. https://api.semanticscholar.org/CorpusID:439683710.1109/CVPR.2018.00479 ↗
17. Wang Y et al (2019) Dynamic graph cnn for learning on point clouds. ACM Trans Graph 38:1–1310.1145/3355089.3356541 ↗
18. Yang Y, Qiu J, Song M, Tao D, Wang X (2020) Distilling knowledge from graph convolutional networks. 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR) 7072–7081. https://api.semanticscholar.org/CorpusID:21471384810.1109/CVPR42600.2020.00710 ↗
19. Rusch TK, Bronstein MM, Mishra S (2023) A survey on oversmoothing in graph neural networks. pp 1–10. arXiv:abs/2303.10993. https://api.semanticscholar.org/CorpusID:257632346
20. Han K, Wang Y, Guo J, Tang Y, Wu E (2022) Vision gnn: an image is worth graph of nodes. In: Proceedings of the 36th international conference on neural information processing systems. pp 1–15
21. Lapenna M et al (2024) Geometric deep learning for enhanced quantitative analysis of microstructures in x-ray computed tomography data. Discover Appl Sci 6:1–910.1007/s42452-024-05985-0 ↗
22. Kipf T, Welling M (2016) Semi-supervised classification with graph convolutional networks. pp 1–14. arXiv:abs/1609.02907. https://api.semanticscholar.org/CorpusID:31442186
23. Zachary WW (1977) An information flow model for conflict and fission in small groups. J Anthropol Res 33:452–47310.1086/jar.33.4.3629752 ↗
24. Buades A, Coll B, Morel J.-M (2005) A non-local algorithm for image denoising. In: Computer society conference on computer vision and pattern recognition, vol 2. pp 60–6510.1109/CVPR.2005.38 ↗
25. Tsamos A, Evsevleev S, Bruno G (2023) Noise and blur removal from corrupted x-ray computed tomography scans: A multilevel and multiscale deep convolutional framework approach with synthetic training data (bam synthcond). Tomography Mater Struct 1–1610.1016/j.tmater.2023.100011 ↗
26. Tsamos A, Evsevleev S, Fioresi R, Faglioni F, Bruno G (2023) A complete strategy to achieve high precision automatic segmentation of challenging experimental x-ray computed tomography data using low-resemblance synthetic training data. Adv Eng Mater 26:1–910.1002/adem.202301030 ↗
27. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, pp 330–371. http://www.deeplearningbook.org
28. Velickovic P (2022) Message passing all the way up. pp 1–10. arXiv:abs/2202.11097. https://api.semanticscholar.org/CorpusID:247026068
29. Veličković P et al (2018) Graph attention networks. In: International conference on learning representations. pp 1–12
30. Keyulu Xu J. L, Weihua Hu, Jegelka S (2019) How powerful are graph neural networks? In: International conference on learning representations. pp 1–17. https://openreview.net/forum?id=ryGs6iA5Km
31. Weisfeiler B, Leman A (1968) The reduction of a graph to a canonical form and the algebra arising during this reduction. Nauchno-Technicheskaya Informatsia 9:1–11
32. Shervashidze Nino VLEJ, Schweitzer P, Kurt M, Borgwardt KM (2011) Weisfeiler-lehman graph kernels. J Mach Learn Res 12:1–23
33. Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st international conference on neural information processing systems. pp 1025–1035
34. He P, Qu A, Xiao S, Ding M (2023) A gnn-based network for tissue semantic segmentation in histopathology image. J Phys: Conf Ser 2504:012047
35. Stefan Elfwing EU, Doya K (2017) Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. pp 1–18. arxiv.org/abs/1702.03118
36. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: 3rd International conference on learning representations. ICLR, pp 1–15
37. Genç A, Kovarik L, Fraser HL (2022) A deep learning approach for semantic segmentation of unbalanced data in electron tomography of catalytic materials. Sci Rep 12:1–1210.1038/s41598-022-16429-3 ↗
38. Thambawita V et al (2022) Singan-seg: Synthetic training data generation for medical image segmentation. PLOS ONE 17:1–2410.1371/journal.pone.0267976 ↗
39. Nguyen Q, Vu T, Tran A, Nguyen K (2023) Diffusion-based synthetic data generation for pixel-level semantic segmentation. Adv Neural Inf Proc Syst 36:1–21
40. Corvi R, Cozzolino D, Poggi G, Nagano K, Verdoliva L (2023) Intriguing properties of synthetic images: from generative adversarial networks to diffusion models. In: 2023 IEEE/CVF Conference on computer vision and pattern recognition workshops (CVPRW). pp 973–982. https://api.semanticscholar.org/CorpusID:25810786910.1109/CVPRW59228.2023.00104 ↗
41. Gao H, Ji S (2022) Graph u-nets. IEEE Trans Pattern Anal Mach Intell 44:4948–4960