Feature Augmentation Based on Pixel-Wise Attention for Rail Defect Detection
Source abstract
Abstract
Image-based rail defect detection could be conceptually defined as an object detection task in computer vision. However, unlike academic object detection tasks, this practical industrial application suffers from two unique challenges, including object ambiguity and insufficient annotations. To overcome these challenges, we introduce the pixel-wise attention mechanism to fully exploit features of annotated defects, and develop a feature augmentation framework to tackle the defect detection problem. The pixel-wise attention is conducted through a learnable pixel-level similarity between input and support features to obtain augmented features. These augmented features contain co-existing information from input images and multi-class support defects. The final output features are augmented and refined by support features, thus endowing the model to distinguish between ambiguous defect patterns based on insufficient annotated samples. Experiments on the rail defect dataset demonstrate that feature augmentation can help balance the sensitivity and robustness of the model. On our collected dataset with eight defected classes, our algorithm achieves 11.32% higher mAP@.5 compared with original YOLOv5 and 4.27% higher mAP@.5 compared with Faster R-CNN.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Infrastructure Maintenance and Monitoring · Industrial Vision Systems and Defect Detection · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Astronautics, Beihang University, Beijing 100191, China
Infrastructure Inspection Research Institute, China Academy of Railway Science Corporation Limited, Beijing 100081, China
Funding
High-speed Rail Joint Fund Projects · Scientific Research Projects of China Academy of Railway Sciences · Science and Technology R&D Program of China Railway Corporation
References (49)
Show source-provided references
1. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
2. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
3. Liu, S., Qi, L., Qin, H., Shi, J., and Jia, J. (2018, January 18–23). Path aggregation network for instance segmentation. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00913 ↗
4. Faster r-cnn: Towards real-time object detection with region proposal networks
5. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
6. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
7. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
8. Railway track fastener defect detection based on image processing and deep learning techniques: A comparative study10.1016/j.engappai.2019.01.008 ↗
9. Hinterstoisser, S., Lepetit, V., Wohlhart, P., and Konolige, K. (2018, January 8–14). On pre-trained image features and synthetic images for deep learning. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany.10.1007/978-3-030-11009-3_42 ↗
10. Automated abnormality detection in lower extremity radiographs using deep learning10.1038/s42256-019-0126-0 ↗
11. Convolutional neural networks for medical image analysis: Full training or fine tuning?10.1109/TMI.2016.2535302 ↗
12. Mnih, V., Heess, N., and Graves, A. (2014). Recurrent models of visual attention. Advances in Neural Information Processing Systems, Curran Associates, Inc.
13. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
14. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, NeurIPS.
15. Hou, R., Chang, H., Ma, B., Shan, S., and Chen, X. (2019). Cross attention network for few-shot classification. arXiv.
16. Hu, H., Gu, J., Zhang, Z., Dai, J., and Wei, Y. (2018, January 18–23). Relation Networks for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00378 ↗
17. Kang, B., Liu, Z., Wang, X., Yu, F., Feng, J., and Darrell, T. (November, January 27). Few-shot object detection via feature reweighting. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea.
18. Fan, Q., Zhuo, W., Tang, C.K., and Tai, Y.W. (2020, January 13–19). Few-shot object detection with attention-RPN and multi-relation detector. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00407 ↗
19. Yan, X., Chen, Z., Xu, A., Wang, X., Liang, X., and Lin, L. (November, January 27). Meta r-cnn: Towards general solver for instance-level low-shot learning. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea.
20. Hu, H., Bai, S., Li, A., Cui, J., and Wang, L. (2021, January 19–25). Dense Relation Distillation with Context-aware Aggregation for Few-Shot Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Virtual.10.1109/CVPR46437.2021.01005 ↗
21. Lienhart, R., and Maydt, J. (2002, January 22–25). An extended set of haar-like features for rapid object detection. Proceedings of the International Conference on Image Processing, Rochester, NY, USA.
22. Mizuno, K., Terachi, Y., Takagi, K., Izumi, S., Kawaguchi, H., and Yoshimoto, M. (2012, January 17–19). Architectural study of HOG feature extraction processor for real-time object detection. Proceedings of the 2012 IEEE Workshop on Signal Processing Systems, Quebec City, QC, Canada.10.1109/SiPS.2012.57 ↗
23. Multi-class adaboost10.4310/SII.2009.v2.n3.a8 ↗
24. Review of object detection methods based on SVM
25. Selective search for object recognition10.1007/s11263-013-0620-5 ↗
26. Zitnick, C.L., and Dollár, P. (2014, January 6–12). Edge boxes: Locating object proposals from edges. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_26 ↗
27. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
28. Durand, T., Mordan, T., Thome, N., and Cord, M. (2017, January 21–26). WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.631 ↗
29. Pan, X., Ren, Y., Sheng, K., Dong, W., Yuan, H., Guo, X., Ma, C., and Xu, C. (2020, January 13–19). Dynamic refinement network for oriented and densely packed object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01122 ↗
30. Xu, B., Liang, H., Liang, R., and Chen, P. (2021, January 2–9). Locate Globally, Segment Locally: A Progressive Architecture With Knowledge Review Network for Salient Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.10.1609/aaai.v35i4.16408 ↗
31. A novel weakly-supervised approach for RGB-D-based nuclear waste object detection10.1109/JSEN.2018.2888815 ↗
32. Ouyang, W., Wang, X., Zhang, C., and Yang, X. (2016, January 27–30). Factors in finetuning deep model for object detection with long-tail distribution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.100 ↗
33. A survey on image data augmentation for deep learning10.1186/s40537-019-0197-0 ↗
34. Person re-identification by camera correlation aware feature augmentation10.1109/TPAMI.2017.2666805 ↗
35. Multi-level semantic feature augmentation for one-shot learning10.1109/TIP.2019.2910052 ↗
36. Northcutt, C.G., Athalye, A., and Mueller, J. (2021). Pervasive label errors in test sets destabilize machine learning benchmarks. arXiv.
37. Choi, J., Elezi, I., Lee, H.J., Farabet, C., and Alvarez, J.M. (2021, January 11–17). Active Learning for Deep Object Detection via Probabilistic Modeling. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Virtual.10.1109/ICCV48922.2021.01010 ↗
38. Dai, X., Chen, Y., Xiao, B., Chen, D., Liu, M., Yuan, L., and Zhang, L. (2021, January 20–25). Dynamic Head: Unifying Object Detection Heads with Attentions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00729 ↗
39. Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., and Girshick, R. (2021, June 20). Detectron2. Available online: https://github.com/facebookresearch/detectron2.
40. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
41. Detnas: Backbone search for object detection
42. ultralytics (2021, September 07). YOLOv5. Available online: https://github.com/ultralytics/yolov5.
43. ultralytics (2022, July 30). YOLOv6. Available online: https://github.com/meituan/YOLOv6.
44. Li, B., Yang, B., Liu, C., Liu, F., Ji, R., and Ye, Q. (2021, January 20–25). Beyond Max-Margin: Class Margin Equilibrium for Few-shot Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00728 ↗
45. Han, G., He, Y., Huang, S., Ma, J., and Chang, S.F. (2021, January 20–25). Query adaptive few-shot object detection with heterogeneous graph convolutional networks. Proceedings of the IEEE/CVF International Conference on Computer Vision, Nashville, TN, USA.10.1109/ICCV48922.2021.00325 ↗
46. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
47. Santur, Y., Karaköse, M., and Akin, E. (2017, January 16–17). A new rail inspection method based on deep learning using laser cameras. Proceedings of the 2017 International Artificial Intelligence and Data Processing Symposium (IDAP), Malatya, Turkey.10.1109/IDAP.2017.8090245 ↗
48. Santur, Y., Karaköse, M., and Akın, E. (2016, January 1–3). Condition monitoring approach using 3D modelling of railway tracks with laser cameras. Proceedings of the International Conference on Advanced Technology &Sciences (ICAT’16), Konya, Turkey.
49. Santur, Y., Karaköse, M., and Akın, E. (2016). Learning based experimental approach for condition monitoring using laser cameras in railway tracks. Int. J. Appl. Math. Electron. Comput., 1–5.10.18100/ijamec.270656 ↗