journal article·2024-04-24Open access

Remote Sensing and Machine Learning for Safer Railways: A Review

Wesam Helmi, Raj Bridgelall, Taraneh Askarzadeh

Applied Sciences· Volume 14· Issue 9

Abstract

Regular railway inspections are crucial for maintaining their safety and efficiency. However, traditional inspection methods are complex and expensive. Consequently, there has been a significant shift toward combining remote sensing (RS) and machine learning (ML) techniques to enhance the efficiency and accuracy of railway defect monitoring while reducing costs. The advantages of RS-ML techniques include their ability to automate and refine inspection processes and address challenges such as image quality and methodological limitations. However, the integration of RS and ML in railway monitoring is an emerging field, with diverse methodologies and outcomes that the research has not yet synthesized. To fill this gap, this study conducted a systematic literature review (SLR) to consolidate the existing research on RS-ML applications in railway inspection. The SLR meticulously compiled and analyzed relevant studies, evaluating the evolution of research trends, methodological approaches, and the geographic distribution of contributions. The findings showed a notable increase in relevant research activity over the last five years, highlighting the growing interest in this realm. The key methodological patterns emphasize the predominance of approaches based on convolutional neural networks, a variant of artificial neural networks, in achieving high levels of precision. These findings serve as a foundational resource for academics, researchers, and practitioners in the fields of computer science, engineering, and transportation to help guide future research directions and foster the development of more efficient, accurate, and cost-effective railway inspection methods.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Remote Sensing and LiDAR Applications · Advanced Measurement and Detection Methods · Automated Road and Building Extraction

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Transportation, Logistics, & Finance, College of Business, North Dakota State University, P.O. Box 6050, Fargo, ND 58108-6050, USA

References (75)

Show source-provided references
1. Aleissaee, A.A., Kumar, A., Anwer, R.M., Khan, S., Cholakkal, H., Xia, G.-S., and Khan, F.S. (2023). Transformers in Remote Sensing: A Survey. Remote Sens., 15.10.3390/rs15071860 ↗
2. Deutschl, E., Gasser, C., Niel, A., and Werschonig, J. (2004, January 14–17). Defect detection on rail surfaces by a vision based system. Proceedings of the IEEE Intelligent Vehicles Symposium, Parma, Italy.
3. Peng, J.-P., Wang, L., Gao, X.-R., Wang, Z.-Y., and Zhao, Q.-K. (2008, January 19–21). Dynamic detection technology for the irregularity state of railway track based on linear array CCD. Proceedings of the AOMATT 2008—4th International Symposium on Advanced Optical Manufacturing, Chengdu, China.10.1117/12.828630 ↗
4. Santur, Y., Karaköse, M., and Akin, E. (2017, January 16–17). A new rail inspection method based on deep learning using laser cameras. Proceedings of the International Artificial Intelligence and Data Processing Symposium (IDAP), Malatya, Turkey.10.1109/IDAP.2017.8090245 ↗
5. Transportation Infrastructure Asset Management using LiDAR Remote Sensing Technology
6. Hu, F., van Leijen, F.J., Chang, L., Wu, J., and Hanssen, R.F. (2019). Monitoring Deformation along Railway Systems Combining Multi-Temporal InSAR and LiDAR Data. Remote Sens., 11.10.3390/rs11192298 ↗
7. Systematic Literature Review of Drone Utility in Railway Condition Monitoring10.1061/JTEPBS.TEENG-7726 ↗
8. Wireless Sensor Networks for Condition Monitoring in the Railway Industry: A Survey10.1109/TITS.2014.2366512 ↗
9. He, R., and Ding, Z. (2019). Applications of Machine Learning in Wireless Communications, IET Digital Library.10.1049/PBTE081E ↗
10. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews10.1186/s13643-021-01626-4 ↗
11. Litmaps (2024, January 23). Litmaps. Available online: https://www.litmaps.com/.
12. Salehi, M., and Wang, Y. (2024, January 23). Rail Network Detection from Aerial Imagery using Deep Learning. Computer Science, Mathematics, 2017. Available online: https://cs231n.stanford.edu/reports/2017/posters/564.pdf.
13. Otero, L.D., Moyou, M., Peter, A., and Otero, C.E. (2018, January 19–22). Towards a Remote Sensing System for Railroad Bridge Inspections: A Concrete Crack Detection Component. Proceedings of the IEEE SoutheastCon, St. Petersburg, FL, USA.
14. Machine Vision Approach for Automating Vegetation Detection on Railway Tracks
15. Mukhamediev, R.I., Symagulov, A., Kuchin, Y., Yakunin, K., and Yelis, M. (2021). From Classical Machine Learning to Deep Neural Networks: A Simplified Scientometric Review. Appl. Sci., 11.10.3390/app11125541 ↗
16. James, G., Witten, D., Hastie, T., and Tibshirani, R. (2013). An Introduction to Statistical Learning, Springer.10.1007/978-1-4614-7138-7 ↗
17. Applying unsupervised machine learning to counterterrorism10.1007/s42001-022-00164-w ↗
18. A Deep Extractor for Visual Rail Surface Inspection10.1109/ACCESS.2021.3055512 ↗
19. Guo, B., Geng, G., Zhu, L., Shi, H., and Yu, Z. (2019). High-Speed Railway Intruding Object Image Generating with Generative Adversarial Networks. Sensors, 19.10.3390/s19143075 ↗
20. Alam Hashmi, M.S., Ibrahim, M., Bajwa, I.S., Siddiqui, H.-U., Rustam, F., Lee, E., and Ashraf, I. (2022). Railway Track Inspection Using Deep Learning Based on Audio to Spectrogram Conversion: An on-the-Fly Approach. Sensors, 22.10.3390/s22051983 ↗
21. Chandran, P., Asber, J., Thiery, F., Odelius, J., and Rantatalo, M. (2021). An Investigation of Railway Fastener Detection Using Image Processing and Augmented Deep Learning. Sustainability, 13.10.3390/su132112051 ↗
22. Fault Detection from Images of Railroad Lines Using the Deep Learning Model Built with the Tensorflow Library10.55525/tjst.1056283 ↗
23. Deep Multitask Learning for Railway Track Inspection10.1109/TITS.2016.2568758 ↗
24. A Big Data Analysis Approach for Rail Failure Risk Assessment10.1111/risa.12836 ↗
25. A Fastener Inspection Method Based on Defective Sample Generation and Deep Convolutional Neural Network10.1109/JSEN.2021.3062021 ↗
26. An in-depth assessment of convolutional neural networks for rail surface defect detection10.33448/rsd-v11i8.30252 ↗
27. Deep learning-based fast detection of apparent concrete crack in slab tracks with dilated convolution10.1016/j.conbuildmat.2022.127157 ↗
28. Fully Decoupled Residual ConvNet for Real-Time Railway Scene Parsing of UAV Aerial Images10.1109/TITS.2021.3134318 ↗
29. Otero, L.D., Peter, A., and Moyou, M. (2016). Rail Safety IDEA Program Remote Sensing with Mobile LiDAR and Imaging Sensors for Railroad Bridge Inspections Final Report for Rail Safety IDEA Project 26, Transportation Research Board.
30. Saha, S., Karmakar, S., and Manna, D. (2022, January 24–26). Analysis of Railroad Track Crack Detection using Computer Vision. Proceedings of the IEEE Interdisciplinary Research in Technology and Management (IRTM), Kolkata, India.10.1109/IRTM54583.2022.9791808 ↗
31. Sánchez-Rodríguez, A., Soilán, M., Cabaleiro, M., and Arias, P. (2019). Automated Inspection of Railway Tunnels’ Power Line Using LiDAR Point Clouds. Remote Sens., 11.10.3390/rs11212567 ↗
32. Prediction of Rail Contact Fatigue on Crossings Using Image Processing and Machine Learning Methods10.1007/s40864-019-0105-0 ↗
33. Memory Linked Anomaly Metric Learning of Thermography Rail Defects Detection System10.1109/JSEN.2021.3112698 ↗
34. Learning Visual Similarity for Inspecting Defective Railway Fasteners10.1109/JSEN.2019.2911015 ↗
35. Chandran, P., Thierry, F., Odelius, J., Famurewa, S.M., Lind, H., and Rantatalo, M. (2021). Supervised Machine Learning Approach for Detecting Missing Clamps in Rail Fastening System from Differential Eddy Current Measurements. Appl. Sci., 11.10.3390/app11094018 ↗
36. Automatic Visual Detection System of Railway Surface Defects with Curvature Filter and Improved Gaussian Mixture Model10.1109/TIM.2018.2803830 ↗
37. Zachar, P., Ostrowski, W., Płatek-Żak, A., and Kurczyński, Z. (2022). The Influence of Point Cloud Accuracy from Image Matching on Automatic Preparation of Training Datasets for Object Detection in UAV Images. ISPRS Int. J. Geo-Inf., 11.10.3390/ijgi11110565 ↗
38. Ultrasonic Guided Waves-Based Monitoring of Rail Head: Laboratory and Field Tests
39. Obstacle detection of rail transit based on deep learning10.1016/j.measurement.2021.109241 ↗
40. Deep Learning for the Detection and Recognition of Rail Defects in Ultrasound B-Scan Images10.1177/03611981211021547 ↗
41. Sharma, R., Patel, K., Shah, S., and Aibin, M. (2022). Aerial Footage Analysis Using Computer Vision for Efficient Detection of Points of Interest Near Railway Tracks. Aerospace, 9.10.3390/aerospace9070370 ↗
42. Multi-Target Defect Identification for Railway Track Line Based on Image Processing and Improved YOLOv3 Model10.1109/ACCESS.2020.2984264 ↗
43. Rail fastener detection of heavy railway based on deep learning10.1016/j.hspr.2022.11.001 ↗
44. MYOLOv3-Tiny: A new convolutional neural network architecture for real-time detection of track fasteners10.1016/j.compind.2020.103303 ↗
45. Zhao, Y., Liu, Z., Yi, D., Yu, X., Sha, X., Li, L., Sun, H., Zhan, Z., and Li, W.J. (2022). A Review on Rail Defect Detection Systems Based on Wireless Sensors. Sensors, 22.10.3390/s22176409 ↗
46. A Review of Research on Detection and Evaluation of the Rail Surface Defects10.12700/APH.19.3.2022.3.14 ↗
47. Recent applications of big data analytics in railway transportation systems: A survey10.1016/j.trc.2018.03.010 ↗
48. Gong, W., Akbar, M.F., Jawad, G.N., Mohamed, M.F.P., and Ab Wahab, M.N. (2022). Nondestructive Testing Technologies for Rail Inspection: A Review. Coatings, 12.10.3390/coatings12111790 ↗
49. Deep Architecture for High-Speed Railway Insulator Surface Defect Detection: Denoising Autoencoder with Multitask Learning10.1109/TIM.2018.2868490 ↗
50. Foreign object detection for railway ballastless trackbeds: A semisupervised learning method10.1016/j.measurement.2022.110757 ↗
51. Automatic railroad track components inspection using real-time instance segmentation10.1111/mice.12625 ↗
52. Railway track fastener defect detection based on image processing and deep learning techniques: A comparative study10.1016/j.engappai.2019.01.008 ↗
53. Application of improved least-square generative adversarial networks for rail crack detection by AE technique10.1016/j.neucom.2018.12.057 ↗
54. Yang, H., Li, X., Guo, Y., and Jia, L. (2022). RT-GAN: GAN Based Architecture for Precise Segmentation of Railway Tracks. Appl. Sci., 12.10.3390/app122312044 ↗
55. A hierarchical learning approach for railway fastener detection using imbalanced samples10.1016/j.measurement.2021.110240 ↗
56. Railroad Surface Defect Segmentation Using a Modified Fully Convolutional Network
57. UAVs in rail damage image diagnostics supported by deep-learning networks10.1515/eng-2021-0033 ↗
58. A Hierarchical Extractor-Based Visual Rail Surface Inspection System10.1109/JSEN.2017.2761858 ↗
59. Densely pyramidal residual network for UAV-based railway images dehazing10.1016/j.neucom.2019.06.076 ↗
60. Unsupervised Saliency Detection of Rail Surface Defects using Stereoscopic Images
61. Real-Time Inspection System for Ballast Railway Fasteners Based on Point Cloud Deep Learning10.1109/ACCESS.2019.2961686 ↗
62. Hybrid deep learning architecture for rail surface segmentation and surface defect detection10.1111/mice.12710 ↗
63. Jin, Y. (2021). Wavelet Scattering and Neural Networks for Railhead Defect Identification. Materials, 14.10.3390/ma14081957 ↗
64. BNSF (2023, January 23). Rail Talk. 27 March 2019. Available online: https://www.bnsf.com/news-media/railtalk/innovation/artificial-intelligence.html.
65. An Efficient Method for High-Speed Railway Dropper Fault Detection Based on Depthwise Separable Convolution10.1109/ACCESS.2019.2942079 ↗
66. Conceptual Understanding of Convolutional Neural Network- A Deep Learning Approach10.1016/j.procs.2018.05.069 ↗
67. Keeling, R., Chhatwal, R., Huber-Fliflet, N., Zhang, J., Wei, F., Zhao, H., Shi, Y., and Qin, H. (2019, January 9–12). Empirical Comparisons of CNN with Other Learning Algorithms for Text Classification in Legal Document Review. Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA.10.1109/BigData47090.2019.9006248 ↗
68. An image-level weakly supervised segmentation method for No-service rail surface defect with size prior10.1016/j.ymssp.2021.108334 ↗
69. SCueU-Net: Efficient Damage Detection Method for Railway Rail10.1109/ACCESS.2020.3007603 ↗
70. Chandran, P., Thiery, F., Odelius, J., Lind, H., and Rantatalo, M. (2022). Unsupervised Machine Learning for Missing Clamp Detection from an In-Service Train Using Differential Eddy Current Sensor. Sustainability, 14.10.3390/su14021035 ↗
71. A Method of Classifying Railway Sleepers and Surface Defects in Real Environment10.1109/JSEN.2020.3026173 ↗
72. Railway Fastener Inspection by Real-Time Machine Vision10.1109/TSMC.2014.2388435 ↗
73. Automatic Defect Detection of Fasteners on the Catenary Support Device Using Deep Convolutional Neural Network10.1109/TIM.2017.2775345 ↗
74. Automated crack severity level detection and classification for ballastless track slab using deep convolutional neural network10.1016/j.autcon.2020.103484 ↗
75. Segmentation of Track Surface Defects Based on Machine Vision and Neural Networks10.1109/JSEN.2021.3133280 ↗