CNN and monocular vision-based natural feature matching for monitoring displacement of segmental retaining wall under real-world lighting conditions and non-stationary camera

Vision-based displacement monitoring of segmental retaining walls (SRWs) faces significant challenges under field conditions, including varying illumination during daytime and nighttime and camera vibrations induced by strong winds. These factors often degrade the performance of natural target (NT) detection and introduce errors in displacement measurements. This study proposes an integrated framework that combines a convolutional neural network (CNN) for block detection with the AKAZE detector for feature matching to monitor SRW displacement under varying illumination and non-stationary camera conditions. Histogram equalization and homography algorithm were employed to enhance image quality and compensate for camera-induced vibrations. A monitoring camera was installed at a field SRW site to acquire real-time images for evaluating the performance of NT detection and displacement measurement under practical conditions. Additionally, laboratory experiments were conducted to validate the effectiveness of the proposed framework. The results demonstrate that the proposed CNN-based method achieves high accuracy in extracting SRW blocks under diverse illumination conditions, with an F1 score of 0.965 ± 0.001. The AKAZE-based feature matching algorithm shows strong capability in matching NTs on the SRW surface. In most cases, the number of inlier matching features extracted from NTs was higher than that obtained from 20 artificial targets (ATs), confirming the feasibility of displacement monitoring without the need for artificial target installation. Additionally, displacement measurement errors were significantly reduced after applying homography-based image alignment to correct non-stationary images. The successful field implementation and validation of the proposed framework demonstrate its strong potential for practical real-time monitoring of SRW displacement. • A CNN and vision-based framework was developed for SRW displacement monitoring under real-world lighting and non-stationary camera conditions. • Histogram equalization significantly enhances feature matching under daytime illumination. • CNN-based SRW block detection model performs excellently on real field images. • AKAZE algorithm reliably matches NTs on the SRW surface. • Displacement measurement errors are significantly reduced after compensating for camera vibrations.