Transformer-based multimodal learning for predicting mechanical properties in heat-treated stainless steel
Source abstract
Abstract
• Innovative Multimodal Learning Approach: Introduces a novel multimodal learning framework utilizing a Transformer-based model to predict the mechanical properties of vacuum carburized martensitic stainless steel. • Integration of Diverse Data Sources: Successfully combines microstructure images, material composition, and process parameters to construct a comprehensive model for predicting hardness and wear resistance. • Enhanced Prediction Accuracy: Achieves significant improvements in prediction accuracy, with an R 2 value of 0.98 and a mean absolute error (MAE) of 5.23 HV for hardness prediction. • Application of Variational Mode Decomposition (VMD) : Implements VMD to process wear curves, effectively reducing noise and enhancing the accuracy of wear performance predictions. • Potential for Broader Applications: Demonstrates the potential of multimodal learning in materials science, offering a robust framework for future advancements in predictive modeling for material design and processing optimization. Accurately predicting mechanical properties of heat-treated materials is critical for intelligent process control and advanced manufacturing. This study proposes a Transformer-based multimodal learning framework for predicting the hardness and wear behavior of carburized steel after vacuum carburizing. By integrating microstructural images, material compositions, and process parameters, the proposed model effectively captures complex cross-modal relationships. Experimental results show that the multimodal model achieves high prediction accuracy, with an R 2 of 0.98 and MAE of 5.23 HV for hardness prediction. Furthermore, Variational Mode Decomposition (VMD) is introduced to preprocess the wear curve, reducing noise and improving the robustness of friction performance prediction. The results demonstrate the effectiveness and generalizability of the proposed approach, offering a practical AI-based solution for intelligent material property evaluation and process optimization.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Hydrogen embrittlement and corrosion behaviors in metals · Non-Destructive Testing Techniques · Microstructure and Mechanical Properties of Steels
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (45)
Show source-provided references
1. Formation and impact of functionally graded buffer layers between martensitic stainless steel and wrought steel substrate by laser metal deposition10.1016/j.matdes.2021.109489 ↗
2. Optimizing the low-pressure carburizing process of 16Cr3NiWMoVNbE gear steel10.1016/j.jmst.2019.02.001 ↗
3. Effects of chromium and carbon content on microstructure and properties of TiC-steel composites10.1016/j.ijrmhm.2017.12.037 ↗
4. Effect of carburization on the microstructure and laser weldability of 316L stainless steel10.5781/JWJ.2023.41.6.7 ↗
5. Understanding the microstructure-properties relationship of low-temperature carburized austenitic stainless steels through EBSD analysis10.1016/j.surfcoat.2017.05.036 ↗
6. Fatigue behaviour and fracture mechanism of a 316 stainless steel hardened by carburizing10.1016/j.ijfatigue.2003.08.024 ↗
7. A computer vision approach for automated analysis and classification of microstructural image data10.1016/j.commatsci.2015.08.011 ↗
8. Image driven machine learning methods for microstructure recognition10.1016/j.commatsci.2016.05.034 ↗
9. Powder-size driven facile microstructure control in powder-fusion metal additive manufacturing processes10.1038/s41467-024-47257-w ↗
10. A new analysis approach based on haralick texture features for the characterization of microstructure on the example of low-alloy steels10.1016/j.matchar.2018.08.009 ↗
11. Effect of carbide precipitation behavior at high temperatures on microstructure and mechanical properties of M50 steel10.1016/j.jmrt.2022.03.021 ↗
12. Machine learning based optimization method for vacuum carburizing process and its application10.20517/jmi.2022.43 ↗
13. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations10.1016/j.jcp.2018.10.045 ↗
14. Analyses of internal structures and defects in materials using physics-informed neural networks
15. A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics10.1016/j.cma.2021.113741 ↗
16. Mechanistic data-driven prediction of as-built mechanical properties in metal additive manufacturing10.1038/s41524-021-00555-z ↗
17. A Wavelet Transform-Assisted Convolutional Neural Network Multi-Model Framework for monitoring Large-Scale Fluorochemical Engineering Processes10.3390/pr8111480 ↗
18. Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties10.1103/PhysRevLett.120.145301 ↗
19. Predicting Materials Properties with Little Data using Shotgun transfer Learning10.1021/acscentsci.9b00804 ↗
20. Transfer learning for materials informatics using crystal graph convolutional neural network10.1016/j.commatsci.2021.110314 ↗
21. Building a quantitative composition-microstructure-property relationship of dual-phase steels via multimodal data mining10.1016/j.actamat.2023.118954 ↗
22. Automated electrosynthesis reaction mining with multimodal large language models (MLLMs)10.1039/D4SC04630G ↗
23. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, (n.d.).
24. Multimodal learning with transformers: a survey10.1109/TPAMI.2023.3275156 ↗
25. Multimodal machine learning: a survey and taxonomy10.1109/TPAMI.2018.2798607 ↗
26. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, N. Houlsby, An image is worth 16x16 words: Transformers for image recognition at scale, (2021).
27. Intelligent fault diagnosis of rolling bearings based on continuous wavelet transform-multiscale feature fusion and improved channel attention mechanism10.17531/ein.2023.1.16 ↗
28. Variational mode decomposition10.1109/TSP.2013.2288675 ↗
29. Predicting compressive strength of consolidated molecular solids using computer vision and deep learning10.1016/j.matdes.2020.108541 ↗
30. Advanced Steel Microstructural Classification by Deep Learning Methods10.1038/s41598-018-20037-5 ↗
31. Unveiling yield strength of metallic materials using physics-enhanced machine learning under diverse experimental conditions10.1016/j.actamat.2024.120046 ↗
32. AI applications through the whole Life Cycle of Material Discovery10.1016/j.matt.2020.06.011 ↗
33. Y. Li, T. Colnaghi, Y. Gong, H. Zhang, Y. Yu, Y. Wei, B. Gan, M. Song, A. Marek, M. Rampp, S. Zhang, Z. Pei, M. Wuttig, S. Ghosh, F. Körmann, J. Neugebauer, Z. Wang, B. Gault, Machine Learning‐Enabled Tomographic Imaging of Chemical Short‐Range Atomic Ordering, Adv. Mater. (2024).10.1002/adma.202407564 ↗
34. Discovering and understanding materials through computation10.1038/s41563-021-01015-1 ↗
35. Scaling deep learning for materials discovery10.1038/s41586-023-06735-9 ↗
36. Harmonizing physical and deep learning modeling: a computationally efficient and interpretable approach for property prediction10.1016/j.scriptamat.2024.116350 ↗
37. Prediction of mechanical properties of composite materials using multimodal fusion learning10.1016/j.sna.2023.114433 ↗
38. Mechanical property prediction of aluminium alloys with varied silicon content using deep learning10.1016/j.actamat.2024.120683 ↗
39. BIG DATA and Data-Driven Intelligent Predictive Algorithms to support creativity in Industrial Engineering10.1016/j.cie.2016.11.005 ↗
40. A. Radford, J.W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, (2021). doi:10.48550/arXiv.2103.00020.
41. W. Kim, B. Son, I. Kim, ViLT: Vision-and-language transformer without convolution or region supervision, (2021). doi:10.48550/ARXIV.2102.03334.
42. R. Hu, A. Singh, UniT: Multimodal Multitask Learning with a Unified Transformer, in: 2021 IEEECVF Int. Conf. Comput. Vis. ICCV, IEEE, Montreal, QC, Canada, 2021: pp. 1419–1429. doi:10.1109/ICCV48922.2021.00147.10.1109/ICCV48922.2021.00147 ↗
43. D. Kiela, S. Bhooshan, H. Firooz, E. Perez, D. Testuggine, Supervised multimodal bitransformers for classifying images and text, (2020). doi:10.48550/arXiv.1909.02950.
44. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, (2015). doi:10.48550/arXiv.1512.03385.10.1109/CVPR.2016.90 ↗
45. K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, (2015). doi:10.48550/arXiv.1409.1556.