Research on a composite detection method of pantograph-catenary arc fault based on the fusion of acoustic imaging

Conventional camera-based methods for pantograph–catenary arc fault detection are susceptible to environmental disturbances, which have a detrimental effect on recognition accuracy. To overcome these limitations, the present study proposes a multimodal recognition network that integrates acoustic and visual imaging to enhance robustness. The approach adopts a feature-level fusion strategy to facilitate precise fault identification under varied conditions. The system’s inputs include acoustic, infrared, and visible-light images. Each modality is processed by convolutional neural networks (CNNs) to extract salient features, followed by fault localisation and classification. To address discrepancies across modalities, the architecture incorporates a Top-k initial query selection module and a multimodal deformable cross-attention module, and employs a cascaded Detection Transformer (DETR) framework to optimise multimodal information fusion and complementarity. In light of the paucity of publicly available datasets, a novel multimodal dataset was devised by simulating pantograph–catenary arc faults, containing acoustic, infrared, and visible-light data. Experiments on this dataset demonstrate that the proposed model attains an accuracy of 84%, surpassing single-modality visible-light methods and combined visible–infrared methods by 15.9% and 8.1%, respectively. The findings corroborate the efficacy of multimodal fusion in enhancing arc fault recognition.