Multimodal prediction model for concrete freeze-thaw damage based on natural language processing and deep neural network

Freeze-thaw damage (FTD) prediction is closely related to the durability design of concrete in cold regions. However, current machine learning models for FTD predominantly rely on numerical data, overlooking crucial textual information regarding the FTD process. This limits the prediction accuracy and interpretability of machine learning. To address this, we constructed a comprehensive dataset comprising 1851 samples extracted from 44 publications, which includes both numerical parameters and textual descriptions (i.e., corrosion environments, experimental processes, morphologies, and FTD mechanisms). A multimodal deep learning (MDL) model that integrated natural language processing (NLP) with deep neural networks (DNN) was then developed to predict FTD. The results show that compared with conventional DNN models, the multi-head self-attention model improves the prediction accuracy of concrete mass loss rate and relative dynamic elastic modulus by 8% and 21%, respectively. The visualization indicates that the improvement in the prediction accuracy of the developed MDL model is attributed to the prior knowledge in the textual information.