journal article·2026-06-28

Logic-guided hierarchical attentive taxonomy network for event-level ultrasonic rail defect detection

Yi Zhao, ZheWen Wang, WenBo Chen, Zheng Shi, ShiXing Zhang

Publisher page

Abstract

Machine learning methods for Real-world Railway ultrasonic testing (UT) face a critical bottleneck rooted in extreme few-shot data scarcity and severe long-tail class distribution, compounded by high signal similarity between defective and healthy regions (especially around bolt holes). These issues trigger model overfitting, unstable convergence and unreliable predictions, resulting in logically contradictory per-frame outputs violating physical domain rules and fragmented predictions that hinder maintenance decision-making. Unlike previous works, we propose the first event-level Railway UT defect detection framework, termed Hierarchical Attentive Taxonomy Network (HAT-Net), which addresses this fundamental issue through two complementary mechanisms. A Taxonomic Consistency Loss enforces ontology-derived mutual-exclusion constraints as a soft training regulariser, eliminating logical violations without inference-time overhead. A Hierarchical Attentive Perception Backbone—dilated residual 1D convolutions with Squeeze-and-Excitation channel attention followed by a BiLSTM smoothing layer – captures long-range temporal context while adaptively reweighting noisy probe channels and outputting event-level predictions. Experiments on continuous multi-channel ultrasonic data show that HAT-Net achieves Event F2 of 0.976, mean Average Precision (mAP) of 0.942 (±0.023) and Constraint Violation Rate (CVR) of 0.000%, outperforming prevalent baselines. Improve detection efficiency while making maintenance work more effective. Robustness studies also demonstrate its relatively stable performance against noise and few-shot settings, indicating improved generalisation.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Ultrasonics and Acoustic Wave Propagation · Machine Learning and Algorithms · Generative Adversarial Networks and Image Synthesis

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Chang’an University

Funding

National Natural Science Foundation of China · Natural Science Foundation of Shaanxi Province

References (51)

Show source-provided references
1. 10.3390/coatings1211179010.3390/coatings12111790 ↗
2. 10.1243/09544097JRRT20910.1243/09544097JRRT209 ↗
3. 10.3390/s2310462710.3390/s23104627 ↗
4. 10.1115/1.403729510.1115/1.4037295 ↗
5. 10.1038/s41598-024-59328-510.1038/s41598-024-59328-5 ↗
6. 10.1016/j.hspr.2025.03.00110.1016/j.hspr.2025.03.001 ↗
7. 10.1007/s13296-024-00927-310.1007/s13296-024-00927-3 ↗
8. 10.1784/insi.2014.56.7.36010.1784/insi.2014.56.7.360 ↗
9. 10.1016/j.apacoust.2021.10832910.1016/j.apacoust.2021.108329 ↗
10. 10.1109/TIM.2018.289005310.1109/TIM.2018.2890053 ↗
11. 10.1016/j.asoc.2021.10770610.1016/j.asoc.2021.107706 ↗
12. 10.1016/j.ijleo.2022.16860710.1016/j.ijleo.2022.168607 ↗
13. Bai S, Kolter JZ, Koltun V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. In: 35th International Conference on Machine Learning (ICML); Stockholm, Sweden. Proceedings of Machine Learning Research; 2018. Available from: https://arxiv.org/abs/1803.01271
14. 10.1162/neco.1997.9.8.173510.1162/neco.1997.9.8.1735 ↗
15. 10.1016/j.neunet.2005.06.04210.1016/j.neunet.2005.06.042 ↗
16. Nie Y, Nguyen NH, Sinclair P, et al. A time series is worth 64 words: long-term forecasting with transformers. In: International Conference on Learning Representations (ICLR); Kigali, Rwanda; 2023 May 1–5. ICLR; 2023. Available from: https://openreview.net/group?id=ICLR.cc/2023/Conference
17. Liu Y, Hu T, Zhang H, et al. Itransformer: inverted transformers are effective for time series forecasting. In: International Conference on Learning Representations (ICLR); 2024 May 7–11; Vienna, Austria. 2024.
18. Xu J, Wu H, Wang J, et al. Anomaly transformer: time series anomaly detection with association discrepancy. In: International Conference on Learning Representations (ICLR); April 25–29; On-line. 2022.
19. 10.1109/TKDE.2013.3910.1109/TKDE.2013.39 ↗
20. Ganchev K, Gillenwater J, Taskar B. Posterior regularization for structured latent variable models. J Mach Learn Res. 2010;11:2001–2049.
21. Hu Z, Ma X, Liu Z, et al. Harnessing deep neural networks with logic rules. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics Long Papers; Berlin, Germany; 2016. Vol. 1. p. 2410–2420.10.18653/v1/P16-1228 ↗
22. Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions. In: International Conference on Learning Representations (ICLR); San Juan (PR);. May; 2016.
23. Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2018 Jun 18–22; Salt Lake City, UT, USA. IEEE Computer Society; 2018. p. 7132–7141. Available from: https://openaccess.thecvf.com/content_cvpr_2018/html/Hu_Squeeze-and-Excitation_Networks_CVPR_2018_paper.html
24. 10.1109/CVPR42600.2020.0115510.1109/CVPR42600.2020.01155 ↗
25. 10.1177/0361198125134223610.1177/03611981251342236 ↗
26. 10.1109/TMAG.2018.287099810.1109/TMAG.2018.2870998 ↗
27. 10.1038/nature1453910.1038/nature14539 ↗
28. 10.1002/stc.250410.1002/stc.2504 ↗
29. 10.3390/app1411461910.3390/app14114619 ↗
30. 10.1609/aaai.v36i8.2088110.1609/aaai.v36i8.20881 ↗
31. Wu H, Hu T, Liu Y, et al. Timesnet: temporal 2d-variation modeling for general time series analysis. In: International Conference on Learning Representations (ICLR); 2023 May 1–5; Kigali, Rwanda. ICLR 2023 Organizing Committee; 2023. Available from: https://openreview.net/forum?id=ju_Uqw384Oq
32. Wang S, Wu H, Shi X, et al. Timemixer: decomposable multiscale mixing for time series forecasting. In: International Conference on Learning Representations (ICLR); 2024 May 7–11; Vienna, Austria. ICLR 2024 Organizing Committee; 2024. Available from: https://openreview.net/forum?id=7oLshfEIC2
33. 10.1007/978-3-030-01234-2_110.1007/978-3-030-01234-2_1 ↗
34. 10.1109/ICCV.2017.32410.1109/ICCV.2017.324 ↗
35. Ridnik T, Ben-Baruch E, Zamir N, et al. Asymmetric loss for multi-label classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV); 2021 Oct 10–17; Montreal, QC, Canada (Virtual). IEEE; 2021. p. 82–91. Available from: https://openaccess.thecvf.com/content/ICCV2021/html/Ridnik_Asymmetric_Loss_for_Multi-Label_Classification_ICCV_2021_paper.html
36. Cui Y, Jia M, Lin TY, et al. Class-balanced loss based on effective number of samples. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); 2019 Jun 16–20; Long Beach. IEEE; 2019. p. 9268–9277. Available from: https://openaccess.thecvf.com/content_CVPR_2019/html/Cui_Class-Balanced_Loss_Based_on_Effective_Number_of_Samples_CVPR_2019_paper.html
37. Lafferty J, McCallum A, Pereira FC. Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th International Conference on Machine Learning (ICML 2001); Williamstown, (MA), USA; 2001. p. 282–289.
38. 10.1561/220000001310.1561/2200000013 ↗
39. 10.1109/TIM.2023.325389510.1109/TIM.2023.3253895 ↗
40. Rocktäschel T, Riedel S. End-to-end differentiable proving. In: NeurIPS 2017; 4-9 December 2017; Long Beach, CA, USA. Vol. 30. Curran Associates; 2017. p. 3791–3803. Available from: https://proceedings.neurips.cc/paper/2017/hash/b2ab001909a8a6f04b51920306046ce5-Abstract.html
41. Morin F, Bengio Y. Hierarchical probabilistic neural network language model. In: Proceedings of the International Workshop on Artificial Intelligence and Statistics (AISTATS); Jan 6–8; Bridgetown, Barbados. Society for Artificial Intelligence and Statistics; 2005. p. 246–252. Available from: https://proceedings.mlr.press/r5/morin05a.html
42. 10.1109/TPAMI.1986.476785110.1109/TPAMI.1986.4767851 ↗
43. Farha YA, Gall JM-T. Multi-stage temporal convolutional network for action segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); Long Beach, (CA), USA; 2019. p. 3575–3584.
44. 10.1007/s11263-009-0275-410.1007/s11263-009-0275-4 ↗
45. Reimers N, Gurevych I. Reporting score distributions makes a difference: performance study of LSTM-networks for sequence tagging. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP); 2017 Sept 9-11; Copenhagen, Denmark. Association for Computational Linguistics; 2017. p. 338–348. Available from: https://aclanthology.org/D17-1035/
46. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. In: NEURIPS 2017; 2017 Dec 4–9; Long Beach. Curran Associates, Inc; 2017. p. 5998–6008.
47. Jocher G Chaurasia A Qiu J. Yolov8: ultralytics yolov8. GitHub repository; 2023. Available from: https://github.com/ultralytics/ultralytics
48. Lin M, Chen Q, Yan S. Network in network. In: Proceedings of the International Conference on Learning Representations (ICLR); 2013 May 2–4; Scottsdale, Arizona, USA. 2013; preprint arXiv:13124400.
49. 10.1109/CVPR.2016.9010.1109/CVPR.2016.90 ↗
50. 10.1016/j.ndteint.2022.10270310.1016/j.ndteint.2022.102703 ↗
51. 10.1109/TUFFC.2023.324896810.1109/TUFFC.2023.3248968 ↗