Real-Time Video Surveillance Based Structural Health Monitoring of Civil Structures Using Artificial Neural Network
Source abstract
Abstract
Modern world’s incessantly increasing outdoor traffic load has eventually led to structural health concern and continuous health monitoring of large scale civil structures such as bridges, roads, highways, etc. In this paper, we propose a computer vision based non-destructive structural health monitoring (SHM) method using high speed camera system combined with the brilliance of artificial intelligence. A number of appreciable SHM techniques had been reported that utilizes wired or wireless smart sensors, but the use of nondestructive techniques, such as, digital high speed imaging were rarely employed for detection of dynamic vibrations of civil structures. In the current research, we have developed a high speed video imaging based structural health monitoring system that utilizes blob detection based motion tracking algorithm. It provides factual information regarding localization and displacement of the target object or an existing feature in the civil structure. The modal parameters were subsequently extracted to analyze the level of severity of structural damage within the civil structures. Also, an artificial neural network is trained to infer the qualitative characteristics of structural vibrations based on vibration intensity and the network inferences can be correlated with the conditions of the structure. The efficacy of our vision system in remote measurement of dynamic displacements was demonstrated through a shaking table and a slip desk experiment. The experimental results demonstrate real-time output with satisfactory performance.
Source: openaire · Quality: candidate unverified · Source record ↗
Research context
OpenAlex topics: Structural Health Monitoring Techniques · Infrastructure Maintenance and Monitoring · Advanced Sensor and Control Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (58)
Show source-provided references
1. Alahi, A., Ortiz, R., Vandergheynst, P.: FREAK: fast retina keypoint. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 510–517 (2012)10.1109/CVPR.2012.6247715 ↗
2. Bao, Y., Tang, Z., Li, H., Zhang, Y.: Computer vision and deep learning-based data anomaly detection method for structural health monitoring. Struct. Health Monit. 18, 401–421 (2018)10.1177/1475921718757405 ↗
3. Bay, H., Ess, A., Tuytelaars, T., Gool, L.J.V.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)10.1016/j.cviu.2007.09.014 ↗
4. Busca, G., Cigada, A., Mazzoleni, P., Zappa, E.: Vibration monitoring of multiple bridge points by means of a unique vision-based measuring system. Exp. Mech. 54(2), 255–271 (2014)10.1007/s11340-013-9784-8 ↗
5. Cao, H., Jebali, C., Kouki, A.B., Thakar, S., Nguyen, C.M., Rao, S., Chiao, J.C.: Development of a laser micro-machined interdigitated capacitive strain sensor for structural health monitoring applications. In: IEEE Sensors, pp. 1741–1744 (2014)10.1109/ICSENS.2014.6985360 ↗
6. Catt, S., Fick, B., Hoskins, M., Praski, J., Baqersad, J.: Development of a semiautonomous drone for structural health monitoring of structures using digital image correlation (DIC). Struct. Health Monit. Photogramm. DIC 6, 49–57 (2019)
7. Cha, Y.J., Chen, J., Büyüköztürk, O.: Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters. Eng. Struct. 132, 300–313 (2017)10.1016/j.engstruct.2016.11.038 ↗
8. Chaabane, M., Mansouri, M., Nounou, H., Nounou, M., Slima, M.B., Hamida, A.B.: Damage detection in structural health monitoring using kernel PLS based GLR. In: International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 1–5 (2017)10.1109/ATSIP.2017.8075555 ↗
9. Chen, J.G., Adams, T.M., Sun, H., Bell, E.S., Büyüköztürk, O.: Camera-based vibration measurement of the world war I memorial bridge in portsmouth, New Hampshire. J. Struct. Eng. 144(11), 04018207 (2018)10.1061/(ASCE)ST.1943-541X.0002203 ↗
10. Chew, Z.J., Ruan, T., Zhu, M., Bafleur, M., Dilhac, J.M.: A multifunctional device as both strain sensor and energy harvester for structural health monitoring. In: IEEE Sensors, pp. 1–3 (2016)10.1109/ICSENS.2016.7808554 ↗
11. Chong, A.K., Al-Baghdadi, J.A., Alshadli, D.: High definition video cameras for measuring movement of vibrating bridge structure. In: International Conference on Vibration and Vibroacoustics, pp. 1–10 (2014)
12. Contreras, W.C., Ziavras, S.G.: Efficient structural health monitoring with wireless sensor networks using a vibration-based frequency domain pattern matching technique. In: IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference, pp. 356–362 (2017)10.1109/UEMCON.2017.8249074 ↗
13. Cooley, J.W., Tukey, J.W.: An algorithm for the machine calculation of complex Fourier series. Math. Comput. 19(90), 297–301 (1965)10.1090/S0025-5718-1965-0178586-1 ↗
14. Daubechies, I.: Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math. 41(7), 909–996 (1988)10.1002/cpa.3160410705 ↗
15. Ding, H., He, Y.T., Jiao, S.B.: Rosette eddy current sensor for structural health monitoring. Appl. Mech. Mater. 330, 430–436 (2013)10.4028/www.scientific.net/AMM.330.430 ↗
16. Dworakowski, Z., Kohut, P., Gallina, A., Holak, K., Uhl, T.: Vision-based algorithms for damage detection and localization in structural health monitoring. Struct. Control Health Monit. 23(1), 35–50 (2016)10.1002/stc.1755 ↗
17. Feng, D., Feng, M.Q.: Vision-based multipoint displacement measurement for structural health monitoring. Struct. Control Health Monit. 23(5), 876–890 (2016)10.1002/stc.1819 ↗
18. Feng, D., Feng, M.Q., Ozer, E., Fukuda, Y.: A vision-based sensor for noncontact structural displacement measurement. Sensors 15(7), 16557–16575 (2015)10.3390/s150716557 ↗
19. Giannelli, P., Bulletti, A., Capineri, L.: Multifunctional piezopolymer film transducer for structural health monitoring applications. IEEE Sens. J. 17(14), 4583–4586 (2017)10.1109/JSEN.2017.2710425 ↗
20. Gul, M., Catbas, F.N.: Structural health monitoring and damage assessment using a novel time series analysis methodology with sensor clustering. J. Sound Vib. 330(6), 1196–1210 (2011)10.1016/j.jsv.2010.09.024 ↗
21. Guo, J., Zhu, C.: Dynamic displacement measurement of large-scale structures based on the lucaskanade template tracking algorithm. Mech. Syst. Signal Process. 66–67, 425–436 (2016)10.1016/j.ymssp.2015.06.004 ↗
22. Harmanci, Y.E., Gülan, U., Holzner, M., Chatzi, E.: A novel approach for 3D-structural identification through video recording: magnified tracking. Sensors 19(5), 1229 (2019)10.3390/s19051229 ↗
23. Hoskere, V., Park, J.W., Yoon, H., Spencer Jr., B.F.: Vision-based modal survey of civil infrastructure using unmanned aerial vehicles. J. Struct. Eng. 145(7), 04019062 (2019)10.1061/(ASCE)ST.1943-541X.0002321 ↗
24. Hu, X., Wang, B., Ji, H.: A wireless sensor network-based structural health monitoring system for highway bridges. Comput. Aided Civil Infrastruct. Eng. 28(3), 193–209 (2013)10.1111/j.1467-8667.2012.00781.x ↗
25. ITU-R: Parameter values for the HDTV standards for production and international programme exchange. ITU-R, Geneva (2002)
26. Judd, J., Schmidt, R., Danforth, M., Branscomb, J., Jung, M.: Radio-frequency identification triggering system for structural health monitoring of highway bridges. In: Conte, J.P., Astroza, R., Benzoni, G., Feltrin, G., Loh, K.J., Moaveni, B. (eds.) Experimental Vibration Analysis for Civil Structures, pp. 118–125. Springer, New York (2018)10.1007/978-3-319-67443-8_9 ↗
27. Kaphle, M., Tan, A.C.C.: Differentiating signals from different sources of acoustic emission for structural health monitoring purposes. In: Shen, G., Wu, Z., Zhang, J. (eds.) Advances in Acoustic Emission Technology, pp. 513–521. Springer, New York (2015)10.1007/978-1-4939-1239-1_48 ↗
28. Khuc, T., Catbas, F.N.: Completely contactless structural health monitoring of real-life structures using cameras and computer vision. Struct. Control Health Monit. 24(1), e1852 (2017)10.1002/stc.1852 ↗
29. Kong, X., Li, J.: Non-contact fatigue crack detection in civil infrastructure through image overlapping and crack breathing sensing. Autom. Constr. 99, 125–139 (2019)10.1016/j.autcon.2018.12.011 ↗
30. Kranz, M.S., English, B.A., Whitley, M.R.: RFID-inspired wireless microsensors for structural health monitoring. In: IEEE Aerospace Conference, pp. 1–7 (2016)10.1109/AERO.2016.7500907 ↗
31. Kypris, O., Markham, A.: 3-D displacement measurement for structural health monitoring using low-frequency magnetic fields. IEEE Sens. J. 17(4), 1165–1174 (2017)10.1109/JSEN.2016.2636451 ↗
32. Liu, X., Cao, J., Guo, P.: SenetSHM: towards practical structural health monitoring using intelligent sensor networks. In: IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom), pp. 416–423 (2016)10.1109/BDCloud-SocialCom-SustainCom.2016.68 ↗
33. Lowe, P.S., Scholehwar, T., Yau, J., Kanfoud, J., Gan, T.H., Selcuk, C.: Flexible shear mode transducer for structural health monitoring using ultrasonic guided waves. IEEE Trans. Ind. Inf. 14(7), 2984–2993 (2018)10.1109/TII.2017.2775343 ↗
34. Lydon, D., Lydon, M., Taylor, S., Del Rincon, J.M., Hester, D., Brownjohn, J.: Development and field testing of a vision-based displacement system using a low cost wireless action camera. Mech. Syst. Signal Process. 121, 343–358 (2019)10.1016/j.ymssp.2018.11.015 ↗
35. Lynch, J.P., Loh, K.J.: A summary review of wireless sensors and sensor networks for structural health monitoring. Shock Vib. Dig. 38(2), 91–128 (2006)10.1177/0583102406061499 ↗
36. Mallat, S.: A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989)10.1109/34.192463 ↗
37. Moll, J., Heftrich, C., Fritzen, C.P.: A rapid and velocity-independent damage localization approach for ultrasonic structural health monitoring. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 59(6), 1309–1315 (2012)10.1109/TUFFC.2012.2322 ↗
38. Oppenheim, A.V.: Discrete-Time Signal Processing. Pearson Education India, Chennai (1999)
39. Oshima, Y., Yamamoto, K., Sugiura, K.: Damage assessment of a bridge based on mode shapes estimated by responses of passing vehicles. Smart Struct. Syst. 13(5), 731–753 (2014)10.12989/sss.2014.13.5.731 ↗
40. Poudel, U.P., Fu, G., Ye, J.: Structural damage detection using digital video imaging technique and wavelet transformation. J. Sound Vib. 286, 869–895 (2005)10.1016/j.jsv.2004.10.043 ↗
41. Ribeiro, D., Calada, R., Ferreira, J., Martins, T.: Non-contact measurement of the dynamic displacement of railway bridges using an advanced video-based system. Eng. Struct. 75, 164–180 (2014)10.1016/j.engstruct.2014.04.051 ↗
42. Sabato, A., Feng, M.Q., Fukuda, Y., Carn, D.L., Fortino, G.: A novel wireless accelerometer board for measuring low-frequency and low-amplitude structural vibration. IEEE Sens. J. 16(9), 2942–2949 (2016)10.1109/JSEN.2016.2522940 ↗
43. Shaladi, R., Alatshan, F., Yang, C.: An overview on the applications of structural health monitoring using wireless sensor networks in bridge engineering. In: International Conference on Advances in Science, Engineering, Technology & Natural Resources, pp. 4–11 (2015)
44. Smyth, A., Wu, M.: Multi-rate Kalman filtering for the data fusion of displacement and acceleration response measurements in dynamic system monitoring. Mech. Syst. Signal Process. 21(2), 706–723 (2007)10.1016/j.ymssp.2006.03.005 ↗
45. Song, Y.Z., Bowen, C.R., Kim, A.H., Nassehi, A., Padget, J., Gathercole, N.: Virtual visual sensors and their application in structural health monitoring. Struct. Health Monit. 13(3), 251–264 (2014)10.1177/1475921714522841 ↗
46. Swit, G., Krampikowska, A., Chinh, L.M.: A prototype system for acoustic emission-based structural health monitoring of my thuan bridge. In: Prognostics and System Health Management Conference (PHM-Chengdu), pp. 1–6 (2016)10.1109/PHM.2016.7819877 ↗
47. Talai, S.M., Desai, D.A., Heyns, P.S.: Infrared thermography applied to the prediction of structural vibration behaviour. IOSR J. Mech. Civil Eng. (IOSR-JMCE) 13(6), 10–17 (2016)
48. Teuscher, C.: FPGA implementations of neural networks. IEEE Trans. Neural Netw. 18(5), 1550 (2007)10.1109/TNN.2007.906886 ↗
49. Tokognon, C.A., Gao, B., Tian, G.Y., Yan, Y.: Structural health monitoring framework based on internet of things: a survey. IEEE Internet of Things J. 4(3), 619–635 (2017)10.1109/JIOT.2017.2664072 ↗
50. Torr, P.H.S., Zisserman, A.: MLESAC: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. 78(1), 138–156 (2000)10.1006/cviu.1999.0832 ↗
51. Wang, P., Yan, Y., Tian, G.Y., Bouzid, O., Ding, Z.: Investigation of wireless sensor networks for structural health monitoring. J. Sens. 2012, 1–7 (2012)10.1155/2012/156329 ↗
52. Wikipedia contributors: Pinhole camera model|Wikipedia, the free encyclopedia. https://en.wikipedia.org/w/index.php?title=Pinhole_camera_model&oldid= 871297383 (2018)
53. Wu, Q., Zhang, Y.D., Amin, M.G., Golato, A., Ahmad, F., Santhanam, S.: Structural health monitoring exploiting MIMO ultrasonic sensing and group sparse Bayesian learning. In: Asilomar Conference on Signals, Systems and Computers, pp. 1162–1166 (2014)10.1109/ACSSC.2014.7094640 ↗
54. Yang, H., Xu, X., Neumann, I.: Laser scanning-based updating of a finite-element model for structural health monitoring. IEEE Sens. J. 16(7), 2100–2104 (2016)10.1109/JSEN.2015.2508965 ↗
55. Ye, X.W., Dong, C.Z., Liu, T.: Force monitoring of steel cables using vision-based sensing technology: methodology and experimental verification. Smart Struct. Syst. 18(3), 585–599 (2016)10.12989/sss.2016.18.3.585 ↗
56. Ye, X.W., Dong, C.Z., Liu, T.: Image-based structural dynamic displacement measurement using different multi-object tracking algorithms. Smart Struct. Syst. 17(6), 856–935 (2016)
57. Zaurin, R., Catbas, F.N.: Computer vision and sensor fusion for structural health monitoring framework with emphasis on unit in uence line analysis. In: Proceedings of the IMAC-XXVII, pp. 1–9 (2009)10.1088/0964-1726/19/1/015019 ↗
58. Zhao, Y., Zhu, Y., Yuan, M., Wang, J., Zhu, S.: A laser-based fiber Bragg grating ultrasonic sensing system for structural health monitoring. IEEE Photonics Technol. Lett. 28(22), 2573–2576 (2016)10.1109/LPT.2016.2605699 ↗