journal article·2024-03Open access

Automated Grain Boundary (GB) Segmentation and Microstructural Analysis in 347H Stainless Steel Using Deep Learning and Multimodal Microscopy

Shoieb Ahmed Chowdhury, M. F. N. Taufique, Jing Wang, Marissa Masden, Madison Wenzlick, Ram Devanathan, Alan L. Schemer-Kohrn, Keerti S. Kappagantula

Abstract

Austenitic 347H stainless steel offers superior mechanical properties and corrosion resistance required for extreme operating conditions such as high temperature. The change in microstructure due to composition and process variations is expected to impact material properties. Identifying microstructural features such as grain boundaries thus becomes an important task in the process-microstructure-properties loop. Applying convolutional neural network (CNN) based deep-learning models is a powerful technique to detect features from material micrographs in an automated manner. Manual labeling of the images for the segmentation task poses a major bottleneck for generating training data and labels in a reliable and reproducible way within a reasonable timeframe. In this study, we attempt to overcome such limitations by utilizing multi-modal microscopy to generate labels directly instead of manual labeling. We combine scanning electron microscopy (SEM) images of 347H stainless steel as training data and electron backscatter diffraction (EBSD) micrographs as pixel-wise labels for grain boundary detection as a semantic segmentation task. We demonstrate that despite producing instrumentation drift during data collection between two modes of microscopy, this method performs comparably to similar segmentation tasks that used manual labeling. Additionally, we find that naïve pixel-wise segmentation results in small gaps and missing boundaries in the predicted grain boundary map. By incorporating topological information during model training, the connectivity of the grain boundary network and segmentation performance is improved. Finally, our approach is validated by accurate computation on downstream tasks of predicting the underlying grain morphology distributions which are the ultimate quantities of interest for microstructural characterization.
Source: openaire · Quality: candidate unverified · Source record ↗

Research context

OpenAlex topics: Hydrogen embrittlement and corrosion behaviors in metals · Corrosion Behavior and Inhibition · Machine Learning in Materials Science

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (54)

Show source-provided references
1. Holmström S, Pohja R, Nurmela A, Moilanen P, Auerkari P (2013) Creep and creep-fatigue behaviour of 316 stainless steel. Procedia Eng 55:160–16410.1016/j.proeng.2013.03.236 ↗
2. He J, Sandström R (2017) Basic modelling of creep rupture in austenitic stainless steels. Theor Appl Fract Mech 89:139–14610.1016/j.tafmec.2017.02.004 ↗
3. Sakthivel T, Vasudevan M, Laha K, Parameswaran P, Chandravathi K, Mathew M, Bhaduri A (2011) Comparison of creep rupture behaviour of type 316L (N) austenitic stainless steel joints welded by TIG and activated TIG welding processes. Mater Sci Eng A 528(22–23):6971–698010.1016/j.msea.2011.05.052 ↗
4. Ozturk K, Zhong Y, Liu Z-K, Luo AA (2003) Creep resistant Mg–Al–Ca alloys: computational thermodynamics and experimental investigation. JOM 55:40–4410.1007/s11837-003-0208-6 ↗
5. Levy A, Pifko A (1981) On computational strategies for problems involving plasticity and creep. Int J Numer Methods Eng 17(5):747–77110.1002/nme.1620170508 ↗
6. Shimada M, Kokawa H, Wang Z, Sato Y, Karibe I (2002) Optimization of grain boundary character distribution for intergranular corrosion resistant 304 stainless steel by twin-induced grain boundary engineering. Acta Mater 50(9):2331–234110.1016/S1359-6454(02)00064-2 ↗
7. Chen Y, Pan S, Zhou M, Yi D, Xu D, Xu Y (2013) Effects of inclusions, grain boundaries and grain orientations on the fatigue crack initiation and propagation behavior of 2524–T3 Al alloy. Mater Sci Eng A 580:150–15810.1016/j.msea.2013.05.053 ↗
8. Sangid MD, Maier HJ, Sehitoglu H (2011) The role of grain boundaries on fatigue crack initiation—an energy approach. Int J Plast 27(5):801–82110.1016/j.ijplas.2010.09.009 ↗
9. Kalinin SV, Ziatdinov M, Spurgeon SR, Ophus C, Stach EA, Susi T, Agar J, Randall J (2022) Deep learning for electron and scanning probe microscopy: from materials design to atomic fabrication. MRS Bull 1–910.1557/s43577-022-00413-3 ↗
10. Holm EA, Cohn R, Gao N, Kitahara AR, Matson TP, Lei B, Yarasi SR (2020) Overview: computer vision and machine learning for microstructural characterization and analysis. Metall Mater Trans A 51:5985–599910.1007/s11661-020-06008-4 ↗
11. Gupta C, Chauhan G, Aiyar HDS (2022) Detection of defective steel surface with image segmentation. In: 2022 international conference on sustainable computing and data communication systems (ICSCDS), IEEE, pp 1578–158210.1109/ICSCDS53736.2022.9760806 ↗
12. Alrfou K, Kordijazi A, Zhao T (2022) Computer vision methods for the microstructural analysis of materials: the state-of-the-art and future perspectives. Preprint http://arxiv.org/abs/2208.04149
13. Azimi SM, Britz D, Engstler M, Fritz M, Mücklich F (2018) Advanced steel microstructural classification by deep learning methods. Sci Rep 8(1):212810.1038/s41598-018-20037-5 ↗
14. Kim H, Inoue J, Kasuya T (2020) Unsupervised microstructure segmentation by mimicking metallurgists’ approach to pattern recognition. Sci Rep 10(1):1783510.1038/s41598-020-74935-8 ↗
15. DeCost BL, Hecht MD, Francis T, Webler BA, Picard YN, Holm EA (2017) UHCSDB: ultrahigh carbon steel micrograph database: tools for exploring large heterogeneous microstructure datasets. Integr Mater Manuf Innov 6:197–20510.1007/s40192-017-0097-0 ↗
16. DeCost BL, Lei B, Francis T, Holm EA (2019) High throughput quantitative metallography for complex microstructures using deep learning: a case study in ultrahigh carbon steel. Microsc Microanal 25(1):21–2910.1017/S1431927618015635 ↗
17. Ajioka F, Wang Z-L, Ogawa T, Adachi Y (2020) Development of high accuracy segmentation model for microstructure of steel by deep learning. ISIJ Int 60(5):954–95910.2355/isijinternational.ISIJINT-2019-568 ↗
18. RonnebergerO, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention—MICCAI 2015: 18th international conference, Munich, Proceedings, Part III 18, Springer, pp 234–24110.1007/978-3-319-24574-4_28 ↗
19. Roberts G, Haile SY, Sainju R, Edwards DJ, Hutchinson B, Zhu Y (2019) Deep learning for semantic segmentation of defects in advanced STEM images of steels. Sci Rep 9(1):1274410.1038/s41598-019-49105-0 ↗
20. Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–140310.1109/ICCV.2015.164 ↗
21. Liu Y, Cheng M-M, Hu X, Wang K, Bai X (2017) Richer convolutional features for edge detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3000–300910.1109/CVPR.2017.622 ↗
22. Zou Q, Zhang Z, Li Q, Qi X, Wang Q, Wang S (2018) Deepcrack: learning hierarchical convolutional features for crack detection. IEEE Trans Image Process 28(3):1498–151210.1109/TIP.2018.2878966 ↗
23. Davies ER (2004) Machine vision: theory, algorithms, practicalities, Elsevier
24. Milesial, Pytorch-UNet (2022). https://github.com/milesial/Pytorch-UNet.
25. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L (2019) Pytorch: an imperative style, high-performance deep learning library. Adv Neural Inf Process Syst 32
26. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–470810.1109/CVPR.2017.243 ↗
27. Novice03, DefectSegNet-pytorch (2021). https://github.com/novice03/DefectSegNet-pytorch
28. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell (6):679–69810.1109/TPAMI.1986.4767851 ↗
29. Sniklaus, pytorch-hed (2022). https://github.com/sniklaus/pytorch-hed/
30. Zou Q (2021) DeepCrack. https://github.com/qinnzou/DeepCrack/
31. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization
32. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
33. Hastie T, Tibshirani R, Friedman JH, Friedman JH (2009) The elements of statistical learning: data mining, inference, and prediction, Springer10.1007/978-0-387-84858-7 ↗
34. Hu X, Li F, Samaras D, Chen C (2019) Topology-preserving deep image segmentation. Adv Neural Inf Process Syst 32
35. Hu X (2021) TopoLoss. https://github.com/HuXiaoling/TopoLoss/blob/master/topoloss_pytorch.py
36. Santos JM, Embrechts M (2009) On the use of the adjusted rand index as a metric for evaluating supervised classification. In: Artificial neural networks–ICANN 2009: 19th international conference, Limassol, Proceedings, Part II 19, Springer, pp 175–18410.1007/978-3-642-04277-5_18 ↗
37. Van der Walt S, Schönberger JL, Nunez-Iglesias J, Boulogne F, Warner JD, Yager N, Gouillart E, Yu T (2014) scikit-image: image processing in Python. PeerJ 2:e45310.7717/peerj.453 ↗
38. Abdi H, Williams LJ (2010) Principal component analysis. Wiley Interdiscip Rev Comput Stat 2(4):433–45910.1002/wics.101 ↗
39. Morris P (2014) Biomedical imaging: applications and advances
40. Cao Y, Liu S, Peng Y, Li J (2020) DenseUNet: densely connected UNet for electron microscopy image segmentation. IET Image Proc 14(12):2682–268910.1049/iet-ipr.2019.1527 ↗
41. Stuckner J, Harder B, Smith TM (2022) Microstructure segmentation with deep learning encoders pre-trained on a large microscopy dataset. npj Comput Mater 8(1):20010.1038/s41524-022-00878-5 ↗
42. Pazdernik K, LaHaye NL, Artman CM, Zhu Y (2020) Microstructural classification of unirradiated LiAlO2 pellets by deep learning methods. Comput Mater Sci 181:10972810.1016/j.commatsci.2020.109728 ↗
43. Edelsbrunner H, Harer JL (2022) Computational topology: an introduction. American Mathematical Society
44. Okai N, Sohda Y (2012) Study on image drift induced by charging during observation by scanning electron microscope. Jpn J Appl Phys 51(6S):06FB1110.1143/JJAP.51.06FB11 ↗
45. Marturi N, Dembélé S, Piat N (2013) Fast image drift compensation in scanning electron microscope using image registration. In: 2013 IEEE international conference on automation science and engineering (CASE), IEEE, pp 807–81210.1109/CoASE.2013.6653936 ↗
46. Li M, Chen D, Liu S, Liu F (2020) Grain boundary detection and second phase segmentation based on multi-task learning and generative adversarial network. Measurement 162:10785710.1016/j.measurement.2020.107857 ↗
47. Thong J, Lee K, Wong W (2001) Reduction of charging effects using vector scanning in the scanning electron microscope. Scanning 23(6):395–40210.1002/sca.4950230606 ↗
48. Stephensen HJT, Darkner S, Sporring J (2020) Restoring drifted electron microscope volumes using synaptic vesicles at sub-pixel accuracy. Commun Biol 3(1):8110.1038/s42003-020-0809-4 ↗
49. Hirth JP (1972) The influence of grain boundaries on mechanical properties. Springer10.1007/BF02661312 ↗
50. Qin W, Li J, Liu Y, Kang J, Zhu L, Shu D, Peng P, She D, Meng D, Li Y (2019) Effects of grain size on tensile property and fracture morphology of 316L stainless steel. Mater Lett 254:116–11910.1016/j.matlet.2019.07.058 ↗
51. Chandra K, Kain V, Tewari R (2013) Microstructural and electrochemical characterisation of heat-treated 347 stainless steel with different phases. Corros Sci 67:118–12910.1016/j.corsci.2012.10.011 ↗
52. McElroy R, Szkopiak Z (1972) Dislocation–substructure–strengthening and mechanical-thermal treatment of metals. Int Metall Rev 17(1):175–20210.1179/095066072790137684 ↗
53. Wang T, Zhu Y, Zhang S, Tang H, Wang H (2015) Grain morphology evolution behavior of titanium alloy components during laser melting deposition additive manufacturing. J Alloy Compd 632:505–51310.1016/j.jallcom.2015.01.256 ↗
54. Endres DM, Schindelin JE (2003) A new metric for probability distributions. IEEE Trans Inf Theory 49(7):1858–186010.1109/TIT.2003.813506 ↗