An anti-interference attention fusion network-based weld penetration state recognition with multimodal data

Accurate identification of weld penetration state serves as an essential basis for evaluating weld quality. Acoustic detection and image detection are two important methods for weld state detection. However, the above methods are easily affected by background noise and environmental factors. Relying solely on a single modality often leads to misjudgement during recognition. Therefore, this paper proposes an anti-interference attention fusion network-based weld penetration state recognition with multimodal data. First, for the acoustic modality, effective signal components are extracted using the Variational Mode Decomposition (VMD) algorithm and visualised via Gramian Angular Summation Field (GASF). Second, for the image modality, exposure is corrected using a decoupling aggregation model, and an edge attention-guided deformable convolution module is introduced to enhance feature extraction from irregular structures in molten pool images. Third, a parallel channel – spatial attention mechanism is employed to strengthen the perception of key features in both modalities, followed through a dual cross-attention fusion module that enables deep semantic alignment and complementary fusion, further improving the discriminative power. Experimental results demonstrate that under undisturbed conditions the proposed method achieves complete recognition of test samples, while maintaining a high accuracy of 93.33% under disturbance, validating its strong effectiveness and robustness.