Automated panoramic infrared thermal image reconstruction of moving railway vehicles coupled with hot box detection by image processing and deep learning-based methods

Ensuring railway safety is a paramount concern, particularly in detecting overheated axle bearings, commonly referred to as hot boxes. In an attempt to improve upon existing operational solutions, the integration of fast infrared (IR) thermal vision technology in the wayside with computer vision has provided new insights for automatically detecting hot-bearing boxes. However, this method involves labour-intensive frame-by-frame processing that can delay critical decision-making. Instead, the present research work shifts focus to the processing of panoramic IR thermal images. Our approach begins with the study and development of a pre-processing technique that employs coarse-to-fine optical flow estimation and image stitching to generate high-resolution panoramic views from sequential IR frames. To enable automatic detection and counting of the hot box, two distinct detection approaches are implemented: custom image processing approach and deep-learning based computer vision approach. The performance of both approaches is evaluated across diverse experimental scenarios, involving both freight and passenger trains and using two distinct IR camera models, cooled and uncooled. Preliminary results are encouraging, demonstrating a clear reduction in the time required for hot box detection and counting, thereby enhancing overall train monitoring efficiency.