journal article·2025-03Open access

Automatic Detection of Dendritic Microstructure Using Computer Vision Deep Learning Models Trained with Phase Field Simulations

A. Viardin, K. Nöth, C. Pickmann, L. Sturz

Abstract

Abstract In this study, a novel approach to analyze the dendritic microstructure in solidification processes is presented, using an innovative method to prepare datasets for deep learning training with minimal human intervention through phase field simulations. This simulation technique, known for its accurate description of dendritic morphologies, enables the creation of comprehensive and precise microstructure datasets. By using advanced deep learning techniques, in particular Faster R-CNN and Mask R-CNN methods, we have successfully automated the detection of dendritic growth in various scenarios. In microgravity and terrestrial solidification experiments with optically transparent alloys, faster R-CNN was particularly effective in identifying loosely connected dendrites in experimental images and showed superior performance over thresholding methods, especially in detecting optically overlapping dendrites. For contiguous dendrites in directionally solidified polycrystalline metal alloys, mask R-CNN proved to be extremely proficient due to its ability to accurately delineate closely spaced dendrites. The use of phase field simulations to generate datasets played a crucial role in training and testing these models. Our research highlights the significant potential of deep learning in describing complex microstructural patterns, contributing to a deeper understanding of the solidification process and its effects on material properties, with the added benefit of facilitating dataset generation through a highly accurate microstructure generation method. Furthermore, this method can be applied to any type of microstructure and to different types of materials, as the phase field simulations can accurately simulate many microstructural properties.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Solidification and crystal growth phenomena · Aluminum Alloy Microstructure Properties · Machine Learning in Materials Science

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (36)

Show source-provided references
1. Holm EA, Cohn R, Gao N, Kitahara AR, Matson TP, Lei B, Yarasi SR (2020) Overview: computer vision and machine learning for microstructural characterization and analysis. Metall Mater Trans A 51(12):5985–5999. https://doi.org/10.1007/s11661-020-06008-410.1007/s11661-020-06008-4 ↗
2. Alrfou K, Zhao T, Kordijazi A (2024) Deep learning methods for microstructural image analysis: the state-of-the-art and future perspectives. Integr Mater Manuf Innov 13:703–731. https://doi.org/10.1007/s40192-024-00369-z10.1007/s40192-024-00369-z ↗
3. Agbozo R, Jin W-Y (2020) Quantitative metallographic analysis of GCr15 microstructure using mask R-CNN. JKSPE 37:361–369. https://doi.org/10.7736/JKSPE.019.14410.7736/JKSPE.019.144 ↗
4. He K, Gkioxari G, Dollár P, Girshick RB (2017) Mask R-CNN, ArXiv. https://doi.org/10.48550/arXiv.1703.0687010.48550/arXiv.1703.06870 ↗
5. Fu L, Yu H, Shah M, Simmons J, Wang S (2020) Crystallographic symmetry for data augmentation in detecting dendrite cores. J Electron Imaging 32:1–7. https://doi.org/10.1117/1.JEI.29.2.02302010.1117/1.JEI.29.2.023020 ↗
6. Ren S, He K, Girshick RB, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks, ArXiv. https://doi.org/10.48550/arXiv.1506.0149710.48550/arXiv.1506.01497 ↗
7. Mulewicz B, Korpala G, Kusiak J, Prahl U (2019) Autonomous interpretation of the microstructure of steels and special alloys. Mater Sci Forum 949:24–31. https://doi.org/10.4028/www.scientific.net/MSF.949.2410.4028/www.scientific.net/MSF.949.24 ↗
8. Elbana R, Mostafa R, Elkeran A (2020) Data processing for automatic classification of spheroidite microstructure using deep learning based on FCNNs. Int J Mech Mechatron Eng 20(3):18–31
9. Banerjee D, Sparks TD (2022) Comparing transfer learning to feature optimization in microstructure classification. iScience 25(2):10377410.1016/j.isci.2022.103774 ↗
10. Liotti E, Arteta C, Zisserman A, Lui A, Lempitsky V, Grant PS (2018) Crystal nucleation in metallic alloys using x-ray radiography and machine learning. Sci Adv. 4(4):eaar400410.1126/sciadv.aar4004 ↗
11. Breumier S, Ostormujof T, Frincu B, Gey N, Couturier A, Loukachenko N, Aba-Perea P-E, Germain L (2022) Leveraging EBSD data by deep learning for bainite, ferrite and martensite segmentation. Mater Charact 186:111805. https://doi.org/10.1016/j.matchar.2021.11163810.1016/j.matchar.2021.111638 ↗
12. Martinez Ostormujof T, Purohit PR, Breumier S, Gey N, Salib M, Germain L (2022) Deep learning for automated phase segmentation in EBSD maps: a case study in dual phase steel microstructures. Mater Charact 184:111638. https://doi.org/10.1016/j.matchar.2021.11163810.1016/j.matchar.2021.111638 ↗
13. Germain L, Sertucha J, Hazotte A, Lacaze J (2024) Classification of graphite particles in metallographic images of cast irons: quantitative image analysis versus deep learning. Mater Charact 217:114333. https://doi.org/10.1016/j.matchar.2024.11433310.1016/j.matchar.2024.114333 ↗
14. Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation, ArXiv, 2015. https://doi.org/10.48550/arXiv.1505.0459710.48550/arXiv.1505.04597 ↗
15. Boettinger WJ, Warren JA, Beckermann C, Karma A (2002) Phase-field simulation of solidification. Annu Rev Mater Res 32:163–194. https://doi.org/10.1146/annurev.matsci.32.101901.15580310.1146/annurev.matsci.32.101901.155803 ↗
16. Kurz W, Fisher DJ (1986) Fundamentals of Solidification. Trans Tech Publications, Zurich
17. Zimmermann G, Hamacher M, Sturz L (2019) In-situ observation of equiaxed dendrites solidifying under reduced gravity conditions during the Texus-55 mission - The Medi-2 experiment, In: Proceedings of the 24rd ESA Symposium on European Rocket and Balloon Programmes and Related Research, ESA-SP 742 p-250-257
18. Sturz L, Hamacher M, Zimmermann G (2023) Effect of composition on nucleation and growth of equiaxed dendrites in the transparent alloy neopentylglycol-(d)camphor during solidification in microgravity conditions. J Cryst Growth 620:127350. https://doi.org/10.1016/j.jcrysgro.2023.12735010.1016/j.jcrysgro.2023.127350 ↗
19. Viardin A, Noth K, Torabi Rad M, Sturz L (2022) Automatic Detection of Equiaxed Dendrites Using Computer Vision Neural Networks, ArXiv. https://doi.org/10.48550/arXiv.2207.0742810.48550/arXiv.2207.07428 ↗
20. Tourret D, Liu H, Llorca J (2022) Phase-field modeling of microstructure evolution: recent applications, perspectives and challenges. Prog Mater Sci 123:100810. https://doi.org/10.1016/j.pmatsci.2021.10081010.1016/j.pmatsci.2021.100810 ↗
21. Cheloee Darabi A, Rastgordani S, Khoshbin M, Guski V, Schmauder S (2023) Hybrid data-driven deep learning framework for material mechanical properties prediction with the focus on dual-phase steel microstructures. Materials 16:447. https://doi.org/10.3390/ma1601044710.3390/ma16010447 ↗
22. Zhuang S, Zhou GD, Huynh P, Areias T, Rabczuk T (2022) Phase field modeling and computer implementation: a review. Eng Fract Mech 262:108234. https://doi.org/10.1016/j.engfracmech.2022.10823410.1016/j.engfracmech.2022.108234 ↗
23. Chen LQ (2002) Phase-field models for microstructure evolution. Annu Rev Mater Res 32:113–140. https://doi.org/10.1146/annurev.matsci.32.112001.13204110.1146/annurev.matsci.32.112001.132041 ↗
24. "MICRESS 7," Available: http://www.micress.de
25. Böttger B, Apel M, Daniels B (2019) Systematic phase-field study on microstructure formation during brazing of mar-M247 with a Si-based AMS4782 filler. Metall Mater Trans A 50:1732–1747. https://doi.org/10.1007/s11661-019-05113-310.1007/s11661-019-05113-3 ↗
26. Boukellal AK, Rouby M, Debierre J-M (2021) Tip dynamics for equiaxed Al-Cu dendrites in thin samples: phase-field study of thermodynamic effects. Comp Mat Sci 186:11005110.1016/j.commatsci.2020.110051 ↗
27. Sturz L, Hamacher M, Eiken J, Zimmermann G, "Multiple Equiaxed Dendrite Interaction Investigated on MASER-13," Proceedings of the 23rd ESA Symposium on European Rocket and Balloon Programmes and Related Research, ESA-SP 741, p. 618-623, (2017)
28. Zimmermann G, Hamacher M, Sturz L, "Columnar-to-Equiaxed Transition in the Transparent Alloy System NPG-DC Observed in the Low-Gravity and Normal Gravity Experiment TRACE-3, In: Proceedings of the 23rd ESA Symposium on European Rocket and Balloon Programmes and Related Research, ESA-SP 741, p. 630-637, (2017)
29. Girshick R, Donahue J, Darrell T, Malik J, "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, p. 580-587, (2014). https://doi.org/10.1109/CVPR.2014.8110.1109/CVPR.2014.81 ↗
30. Girshick RB (2015) Fast R-CNN, ArXiv. https://doi.org/10.48550/arXiv.1504.0808310.48550/arXiv.1504.08083 ↗
31. Viardin A, Zaloznik M, Souhar Y, Apel M, Combeau H (2017) Mesoscopic modeling of spacing and grain selection in columnar dendritic solidification: envelope versus phase-field model. Acta Mater 122:386–399. https://doi.org/10.1016/j.actamat.2016.10.00410.1016/j.actamat.2016.10.004 ↗
32. Wu Y, Kirillov A, Massa F, Lo W-Y, Girshick R Detectron2. https://github.com/facebookresearch/detectron2
33. ParaView https://www.paraview.org
34. OpenMMLab (2022) https://github.com/open-mmlab
35. Bewley A (2023) SORT A simple online and realtime tracking algorithm for 2D multiple object tracking in video sequences. https://github.com/abewley/sort
36. Lin T-Y, Maire M, Belongie SJ, Bourdev LD, Girshick RB, Hays J, Perona P, Ramanan D, Dollar P, Zitnick CL (2014) Microsoft COCO: common objects in context, ArXiv. https://doi.org/10.48550/arXiv.1405.031210.48550/arXiv.1405.0312 ↗