A Review of Deep Learning Applications for Railway Safety
Source abstract
Abstract
Railways speedily transport many people and goods nationwide, so railway accidents can pose immense damage. However, the infrastructure of railways is so complex that its maintenance is challenging and expensive. Therefore, using artificial intelligence for railway safety has attracted many researchers. This paper examines artificial intelligence applications for railway safety, mainly focusing on deep learning approaches. This paper first introduces deep learning methods widely used for railway safety. Then, we investigated and classified earlier studies into four representative application areas: (1) railway infrastructure (catenary, surface, components, and geometry), (2) train body and bogie (door, wheel, suspension, bearing, etc.), (3) operation (railway detection, railroad trespassing, wind risk, train running safety, etc.), and (4) station (air quality control, accident prevention, etc.). We present fundamental problems and popular approaches for each application area. Finally, based on the literature reviews, we discuss the opportunities and challenges of artificial intelligence for railway safety.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Railway Engineering and Dynamics · Hand Gesture Recognition Systems · Occupational Health and Safety Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Human-Computer Interaction, Hanyang University, Ansan-si 15588, Korea
Railroad Structure Research Team, Korea Railroad Research Institute (KRRI), Uiwang-si 16105, Korea
Department of Applied Artificial Intelligence, Hanyang University, Ansan-si 15588, Korea
Funding
Korea Railroad Research Institute · Information & communications Technology Planning & Evaluation (IITP)
References (94)
Show source-provided references
1. Nilsson, N.J., and Nilsson, N.J. (1998). Artificial Intelligence: A New Synthesis, Morgan Kaufmann.
2. McCarthy, J. (2022, September 29). What Is Artificial Intelligence. Available online: http://www-formal.stanford.edu/jmc/whatisai.html.
3. Artificial intelligence and natural man
4. Akhila, C., Diamond, C.A., and Posonia, A.M. (2021, January 6–8). Convolutional Neural network based Online Rail surface Crack Detection. Proceedings of the 2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India.10.1109/ICICCS51141.2021.9432080 ↗
5. Railroad Surface Defect Segmentation Using a Modified Fully Convolutional Network
6. Tsunashima, H. (2019). Condition monitoring of railway tracks from car-body vibration using a machine learning technique. Appl. Sci., 9.10.3390/app9132734 ↗
7. Yang, C., Sun, Y., Ladubec, C., and Liu, Y. (2020). Developing machine learning-based models for railway inspection. Appl. Sci., 11.10.3390/app11010013 ↗
8. Fast classification for rail defect depths using a hybrid intelligent method10.1016/j.ijleo.2018.11.053 ↗
9. Suwansin, W., and Phasukkit, P. (2021). Deep learning-based acoustic emission scheme for nondestructive localization of cracks in train rails under a load. Sensors, 21.10.3390/s21010272 ↗
10. A deep learning approach towards railway safety risk assessment10.1109/ACCESS.2020.2997946 ↗
11. A literature review of Artificial Intelligence applications in railway systems10.1016/j.trc.2022.103679 ↗
12. A review of applications of visual inspection technology based on image processing in the railway industry10.1093/tse/tdz007 ↗
13. Recent applications of big data analytics in railway transportation systems: A survey10.1016/j.trc.2018.03.010 ↗
14. Machine vision-based surface crack analysis for transportation infrastructure10.1016/j.autcon.2021.103973 ↗
15. A taxonomy of railway track maintenance planning and scheduling: A review and research trends10.1016/j.ress.2021.107827 ↗
16. A review on artificial intelligence in high-speed rail10.1093/tse/tdaa022 ↗
17. Train dispatching management with data-driven approaches: A comprehensive review and appraisal10.1109/ACCESS.2019.2935106 ↗
18. Chenariyan Nakhaee, M., Hiemstra, D., Stoelinga, M., and Noort, M.v. (2019, January 4–6). The recent applications of machine learning in rail track maintenance: A survey. Proceedings of the International Conference on Reliability, Safety, and Security of Railway Systems, Lille, France.10.1007/978-3-030-18744-6_6 ↗
19. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
20. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
21. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
22. Res2net: A new multi-scale backbone architecture10.1109/TPAMI.2019.2938758 ↗
23. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
24. Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7–12). Faster R-CNN: Towards real-time object detection with region proposal networks. Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, QC, Canada.
25. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
26. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
27. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
28. Bolya, D., Zhou, C., Xiao, F., and Lee, Y.J. (November, January 27). Yolact: Real-time instance segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.
29. Mika, S., Ratsch, G., Weston, J., Scholkopf, B., and Mullers, K.R. (1999, January 25). Fisher discriminant analysis with kernels. Proceedings of the Neural Networks for Signal Processing IX: 1999 IEEE Signal Processing Society Workshop (cat. no. 98th8468), Madison, WI, USA.
30. Canonical correlation analysis: An overview with application to learning methods10.1162/0899766042321814 ↗
31. Golub, G.H., and Van Loan, C.F. (2013). Matrix Computations, JHU Press.10.56021/9781421407944 ↗
32. A global geometric framework for nonlinear dimensionality reduction10.1126/science.290.5500.2319 ↗
33. Nonlinear dimensionality reduction by locally linear embedding10.1126/science.290.5500.2323 ↗
34. Nonlinear principal component analysis using autoassociative neural networks10.1002/aic.690370209 ↗
35. Training restricted Boltzmann machines: An introduction10.1016/j.patcog.2013.05.025 ↗
36. Deep belief networks10.4249/scholarpedia.5947 ↗
37. Deep architecture for high-speed railway insulator surface defect detection: Denoising autoencoder with multitask learning10.1109/TIM.2018.2868490 ↗
38. An improved faster R-CNN for high-speed railway dropper detection10.1109/ACCESS.2020.3000506 ↗
39. Deep learning-based visual ensemble method for high-speed railway catenary clevis fracture detection10.1016/j.neucom.2018.10.107 ↗
40. A defect-detection method of split pins in the catenary fastening devices of high-speed railway based on deep learning10.1109/TIM.2020.3006324 ↗
41. Roy Choudhury, A., Vanguri, R., Jambawalikar, S.R., and Kumar, P. (2018, January 16). Segmentation of brain tumors using DeepLabv3+. Proceedings of the International MICCAI Brainlesion Workshop, Granada, Spain.10.1007/978-3-030-11726-9_14 ↗
42. Kurth, T., Treichler, S., Romero, J., Mudigonda, M., Luehr, N., Phillips, E., Mahesh, A., Matheson, M., Deslippe, J., and Fatica, M. (2018, January 11–16). Exascale deep learning for climate analytics. Proceedings of the SC18: International Conference for High Performance Computing, Networking, Storage and Analysis, Dallas, TX, USA.10.1109/SC.2018.00054 ↗
43. Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8–14). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European conference on computer vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_49 ↗
44. Fault diagnosis based on deep learning for current-carrying ring of catenary system in sustainable railway transportation10.1016/j.asoc.2020.106907 ↗
45. Santur, Y., Karaköse, M., and Akin, E. (2016, January 1–3). Random forest based diagnosis approach for rail fault inspection in railways. Proceedings of the 2016 National Conference on Electrical, Electronics and Biomedical Engineering (ELECO), Bursa, Turkey.
46. Faghih-Roohi, S., Hajizadeh, S., Núñez, A., Babuska, R., and De Schutter, B. (2016, January 24–29). Deep convolutional neural networks for detection of rail surface defects. Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada.10.1109/IJCNN.2016.7727522 ↗
47. Yanan, S., Hui, Z., Li, L., and Hang, Z. (December, January 30). Rail surface defect detection method based on YOLOv3 deep learning networks. Proceedings of the 2018 Chinese Automation Congress (CAC), Xi’an, China.10.1109/CAC.2018.8623082 ↗
48. Yuan, H., Chen, H., Liu, S., Lin, J., and Luo, X. (2019, January 14–17). A deep convolutional neural network for detection of rail surface defect. Proceedings of the 2019 IEEE Vehicle Power and Propulsion Conference (VPPC), Hanoi, Vietnam.10.1109/VPPC46532.2019.8952236 ↗
49. Shang, L., Yang, Q., Wang, J., Li, S., and Lei, W. (2018, January 11–14). Detection of rail surface defects based on CNN image recognition and classification. Proceedings of the 2018 20th International Conference on Advanced Communication Technology (ICACT), Chuncheon, Korea.10.23919/ICACT.2018.8323641 ↗
50. Liang, Z., Zhang, H., Liu, L., He, Z., and Zheng, K. (2018, January 4–8). Defect detection of rail surface with deep convolutional neural networks. Proceedings of the 2018 13th World Congress on Intelligent Control and Automation (WCICA), Changsha, China.10.1109/WCICA.2018.8630525 ↗
51. An unsupervised method based on convolutional variational auto-encoder and anomaly detection algorithms for light rail squat localization10.1016/j.conbuildmat.2021.125563 ↗
52. Prediction of wheel and rail wear under different contact conditions using artificial neural networks10.1016/j.wear.2018.01.007 ↗
53. Hybrid deep learning architecture for rail surface segmentation and surface defect detection10.1111/mice.12710 ↗
54. Two deep learning networks for rail surface defect inspection of limited samples with line-level label10.1109/TII.2020.3045196 ↗
55. Semi-supervised rail defect detection from imbalanced image data10.1016/j.ifacol.2016.07.014 ↗
56. Santur, Y., Karaköse, M., and Akin, E. (2017, January 16–17). A new rail inspection method based on deep learning using laser cameras. Proceedings of the 2017 International Artificial Intelligence and Data Processing Symposium (IDAP), Malatya, Turkey.10.1109/IDAP.2017.8090245 ↗
57. Development of a tram track degradation prediction model based on the acceleration data10.1080/15732479.2019.1615963 ↗
58. Automatic railroad track components inspection using real-time instance segmentation10.1111/mice.12625 ↗
59. Deep multitask learning for railway track inspection10.1109/TITS.2016.2568758 ↗
60. Detection and severity evaluation of combined rail defects using deep learning10.3390/vibration4020022 ↗
61. Sun, Y., Liu, Y., and Yang, C. (2019, January 22–26). Railway joint detection using deep convolutional neural networks. Proceedings of the 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE), Vancouver, BC, Canada.10.1109/COASE.2019.8843245 ↗
62. Chandran, P., Thierry, F., Odelius, J., Famurewa, S.M., Lind, H., and Rantatalo, M. (2021). Supervised Machine Learning Approach for Detecting Missing Clamps in Rail Fastening System from Differential Eddy Current Measurements. Appl. Sci., 11.10.3390/app11094018 ↗
63. Unsupervised machine learning techniques to prevent faults in railroad switch machines10.1016/j.ijcip.2021.100423 ↗
64. Guo, G., Cui, X., and Du, B. (2021). Random–Forest Machine Learning Approach for High–Speed Railway Track Slab Deformation Identification Using Track-Side Vibration Monitoring. Appl. Sci., 11.10.3390/app11114756 ↗
65. Deep learning for track quality evaluation of high-speed railway based on vehicle-body vibration prediction10.1109/ACCESS.2019.2960537 ↗
66. Hao, X., Yang, J., Yang, F., Sun, X., Hou, Y., and Wang, J. (2022). Track geometry estimation from vehicle–body acceleration for high-speed railway using deep learning technique. Veh. Syst. Dyn., 1–21.10.1080/00423114.2022.2037669 ↗
67. Ham, S., Han, S.Y., Kim, S., Park, H.J., Park, K.J., and Choi, J.H. (2019). A comparative study of fault diagnosis for train door system: Traditional versus deep learning approaches. Sensors, 19.10.3390/s19235160 ↗
68. Wheel defect detection with machine learning10.1109/TITS.2017.2720721 ↗
69. Deep learning based virtual point tracking for real-time target-less dynamic displacement measurement in railway applications10.1016/j.ymssp.2021.108482 ↗
70. Detecting Unexpected Faults of High-Speed Train Bogie Based on Bayesian Deep Learning10.1109/TVT.2020.3048027 ↗
71. Xie, J., Li, T., Yang, Y., and Jin, W. (2014, January 6–11). Learning features from high speed train vibration signals with deep belief networks. Proceedings of the 2014 International Joint Conference on Neural Networks (IJCNN), Beijing, China.10.1109/IJCNN.2014.6889729 ↗
72. Xu, G., Liu, M., Jiang, Z., Söffker, D., and Shen, W. (2019). Bearing fault diagnosis method based on deep convolutional neural network and random forest ensemble learning. Sensors, 19.10.3390/s19051088 ↗
73. Deep learning based approach for bearing fault diagnosis10.1109/TIA.2017.2661250 ↗
74. Fault diagnosis on the bearing of traction motor in high-speed trains based on deep learning10.1016/j.aej.2020.10.044 ↗
75. Zhan, Y., Linb, K., Zhan, H., Guo, Y., and Sun, G. (2018, January 7–10). A unified framework for fault detection of freight train images under complex environment. Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.10.1109/ICIP.2018.8451188 ↗
76. Automatic multi-fault recognition in TFDS based on convolutional neural network10.1016/j.neucom.2016.10.018 ↗
77. A hierarchical features-based model for freight train defect inspection10.1109/JSEN.2019.2954124 ↗
78. Fault detection of railway freight cars mechanical components based on multi-feature fusion convolutional neural network10.1007/s13042-021-01274-z ↗
79. Artificial intelligence-aided automated detection of railroad trespassing10.1177/0361198119846468 ↗
80. Gao, H., Huang, Y., Li, H., and Zhang, Q. (2021, January 14–16). Multi-Sensor Fusion Perception System in Train. Proceedings of the 2021 IEEE 10th Data Driven Control and Learning Systems Conference (DDCLS), Suzhou, China.10.1109/DDCLS52934.2021.9455542 ↗
81. Efficient rail area detection using convolutional neural network10.1109/ACCESS.2018.2883704 ↗
82. RailNet: A segmentation network for railroad detection10.1109/ACCESS.2019.2945633 ↗
83. Short-Term Strong Wind Risk Prediction for High-Speed Railway10.1109/TITS.2021.3058608 ↗
84. Train running safety on non-ballasted bridges10.1080/23248378.2018.1503975 ↗
85. Early-warning method of train running safety of a high-speed railway bridge based on transverse vibration monitoring
86. Evaluation of Train Running Safety for Direct Fixation Concrete Track on Light Rapid Transit
87. Assessment of train running safety, ride comfort and track serviceability at transition between floating slab track and conventional concrete track10.7782/JKSR.2012.15.1.048 ↗
88. Lee, H., Han, S.Y., Park, K., Lee, H., and Kwon, T. (2021). Real-Time Hybrid Deep Learning-Based Train Running Safety Prediction Framework of Railway Vehicle. Machines, 9.10.3390/machines9070130 ↗
89. Heidarysafa, M., Kowsari, K., Barnes, L., and Brown, D. (2018, January 17–20). Analysis of railway accidents’ narratives using deep learning. Proceedings of the 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA.10.1109/ICMLA.2018.00235 ↗
90. Soft sensor validation for monitoring and resilient control of sequential subway indoor air quality through memory-gated recurrent neural networks-based autoencoders10.1016/j.conengprac.2020.104330 ↗
91. A deep learning approach for multi-attribute data: A study of train delay prediction in railway systems10.1016/j.ins.2019.12.053 ↗
92. ST-LSTM: A deep learning approach combined spatio-temporal features for short-term forecast in rail transit10.1155/2019/8392592 ↗
93. Train Time Delay Prediction for High-Speed Train Dispatching Based on Spatio-Temporal Graph Convolutional Network10.1109/TITS.2021.3097064 ↗
94. Wolf, J., Richter, R., and Döllner, J. (2021, January 8–10). Asset Detection in Railroad Environments using Deep Learning-based Scanline Analysis. Proceedings of the VISAPP 2021: 16th International Conference on Computer Vision Theory and Applications, Virtual.10.5220/0010314704650470 ↗