Steel Surface Defect Recognition: A Survey
Source abstract
Abstract
Steel surface defect recognition is an important part of industrial product surface defect detection, which has attracted more and more attention in recent years. In the development of steel surface defect recognition technology, there has been a development process from manual detection to automatic detection based on the traditional machine learning algorithm, and subsequently to automatic detection based on the deep learning algorithm. In this paper, we discuss the key hardware of steel surface defect detection systems and offer suggestions for related options; second, we present a literature review of the algorithms related to steel surface defect recognition, which includes traditional machine learning algorithms based on texture features and shape features as well as supervised, unsupervised, and weakly supervised deep learning algorithms (Incomplete supervision, inexact supervision, imprecise supervision). In addition, some common datasets and algorithm performance evaluation metrics in the field of steel surface defect recognition are summarized. Finally, we discuss the challenges of the current steel surface defect recognition algorithms and the corresponding solutions, and our future work focus is explained.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Surface Roughness and Optical Measurements · Welding Techniques and Residual Stresses
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Software, Shenyang University of Technology, Shenyang 110870, China
School of Mechanical Engineering & Automation, Northeastern University, Shenyang 110819, China
Funding
Liaoning Provincial Department of Education Scientific Research Project
References (100+)
Show source-provided references
1. Basson, E. (2022, April 30). World Steel in Figures 2022. Available online: https://worldsteel.org/steel-topics/statistics/world-steel-in-figures-2022/.
2. Synthetic data augmentation for surface defect detection and classification using deep learning10.1007/s10845-020-01710-x ↗
3. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features10.1109/TIM.2019.2915404 ↗
4. Automated Visual Defect Classification for Flat Steel Surface: A Survey10.1109/TIM.2020.3030167 ↗
5. He, Y., Wen, X., and Xu, J. (2022). A Semi-Supervised Inspection Approach of Textured Surface Defects under Limited Labeled Samples. Coatings, 12.10.3390/coatings12111707 ↗
6. Ma, S., Song, K., Niu, M., Tian, H., and Yan, Y. (2022). Cross-scale Fusion and Domain Adversarial Network for Generalizable Rail Surface Defect Segmentation on Unseen Datasets. J. Intell. Manuf., 1–20.10.1007/s10845-022-02051-7 ↗
7. Wan, C., Ma, S., and Song, K. (2022). TSSTNet: A Two-Stream Swin Transformer Network for Salient Object Detection of No-Service Rail Surface Defects. Coatings, 12.10.3390/coatings12111730 ↗
8. Li, Z., Tian, X., Liu, X., Liu, Y., and Shi, X. (2022). A Two-Stage Industrial Defect Detection Framework Based on Improved-YOLOv5 and Optimized-Inception-ResnetV2 Models. Appl. Sci., 12.10.3390/app12020834 ↗
9. Song, K., Wang, J., Bao, Y., Huang, L., and Yan, Y. (2022). A Novel Visible-Depth-Thermal Image Dataset of Salient Object Detection for Robotic Visual Perception. IEEE/ASME Trans. Mechatron.10.1109/TMECH.2022.3215909 ↗
10. Sun, G., Huang, D., Cheng, L., Jia, J., Xiong, C., and Zhang, Y. (2022). Efficient and Lightweight Framework for Real-Time Ore Image Segmentation Based on Deep Learning. Minerals, 12.10.3390/min12050526 ↗
11. Review of vision-based steel surface inspection systems10.1186/1687-5281-2014-50 ↗
12. (2022, June 13). Verlag Stahleisen GmbH, Germany. Available online: www.stahleisen.de.
13. Robust real-time face detection10.1023/B:VISI.0000013087.49260.fb ↗
14. Histograms of oriented gradients for human detection10.1109/CVPR.2005.177 ↗
15. Object detection with discriminatively trained part-based models10.1109/TPAMI.2009.167 ↗
16. Imagenet classification with deep convolutional neural networks10.1145/3065386 ↗
17. Generative adversarial networks10.1145/3422622 ↗
18. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
19. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
20. Spatial pyramid pooling in deep convolutional networks for visual recognition10.1109/TPAMI.2015.2389824 ↗
21. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
22. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
23. Girshick, R. (2015, January 7–13). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
24. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). Ssd: Single shot multibox detector. European Conference on Computer Vision, Springer.10.1007/978-3-319-46448-0_2 ↗
25. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and<0.5 MB model size. arXiv.
26. Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster r-cnn: Towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst., 28.
27. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
28. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
29. Cai, Z., and Vasconcelos, N. (2018, January 18–22). Cascade r-cnn: Delving into high quality object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Los Alamitos, CA, USA.10.1109/CVPR.2018.00644 ↗
30. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
31. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
32. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–22). Shufflenet: An extremely efficient convolutional neural network for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Los Alamitos, CA, USA.10.1109/CVPR.2018.00716 ↗
33. Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (2019, January 16–20). Centernet: Keypoint triplets for object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Long Beach, CA, USA.10.1109/ICCV.2019.00667 ↗
34. Recent advances in surface defect inspection of industrial products using deep learning techniques10.1007/s00170-021-06592-8 ↗
35. Wan, X., Zhang, X., and Liu, L. (2021). An Improved VGG19 Transfer Learning Strip Steel Surface Defect Recognition Deep Neural Network Based on Few Samples and Imbalanced Datasets. Appl. Sci., 11.10.3390/app11062606 ↗
36. Wang, S., Xia, X., Ye, L., and Yang, B. (2021). Automatic detection and classification of steel surface defect using deep convolutional neural networks. Metals, 11.10.3390/met11030388 ↗
37. Dual attention deep learning network for automatic steel surface defect segmentation10.1111/mice.12792 ↗
38. Steger, C., Ulrich, M., and Wiedemann, C. (2018). Machine Vision Algorithms and Applications, John Wiley & Sons.
39. Hornberg, A. (2017). Handbook of Machine and Computer Vision: The Guide for Developers and User, John Wiley & Sons.10.1002/9783527413409 ↗
40. Deep-learning-based small surface defect detection via an exaggerated local variation-based generative adversarial network10.1109/TII.2019.2945403 ↗
41. Improved Sobel algorithm for defect detection of rail surfaces with enhanced efficiency and accuracy10.1007/s11771-016-3350-3 ↗
42. Image data assessment approach for deep learning-based metal surface defect-detection systems10.1109/ACCESS.2021.3068256 ↗
43. Shreya, S.R., Priya, C.S., and Rajeshware, G.S. (2016, January 16–18). Design of machine vision system for high speed manufacturing environments. Proceedings of the 2016 IEEE Annual India Conference (INDICON), Bangalore, India.10.1109/INDICON.2016.7838973 ↗
44. Chen, Y., Ding, Y., Zhao, F., Zhang, E., Wu, Z., and Shao, L. (2021). Surface defect detection methods for industrial products: A review. Appl. Sci., 11.10.3390/app11167657 ↗
45. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects10.1016/j.apsusc.2013.09.002 ↗
46. Strip steel surface defect recognition based on novel feature extraction and enhanced least squares twin support vector machine10.2355/isijinternational.54.1638 ↗
47. Invariant feature extraction method based on smoothed local binary pattern for strip steel surface defect10.2355/isijinternational.ISIJINT-2015-201 ↗
48. Automatic image thresholding using Otsu’s method and entropy weighting scheme for surface defect detection10.1007/s00500-017-2709-1 ↗
49. Surface defect classification for hot-rolled steel strips by selectively dominant local binary patterns10.1109/ACCESS.2019.2898215 ↗
50. Distributed defect recognition on steel surfaces using an improved random forest algorithm with optimal multi-feature-set fusion10.1007/s11042-017-5238-0 ↗
51. Steel surface defect classification based on discriminant manifold regularized local descriptor10.1109/ACCESS.2018.2881962 ↗
52. Generalized Completed Local Binary Patterns for Time-Efficient Steel Surface Defect Classification
53. Liu, Y., Xu, K., and Xu, J. (2019). An improved MB-LBP defect recognition approach for the surface of steel plates. Appl. Sci., 9.10.3390/app9204222 ↗
54. Application of multi-scale feature extraction to surface defect classification of hot-rolled steels10.1007/s12613-013-0690-y ↗
55. Detection of periodic defects using dual-light switching lighting method on the surface of thick plates10.2355/isijinternational.ISIJINT-2015-053 ↗
56. Application of Shearlet transform to classification of surface defects for metals10.1016/j.imavis.2015.01.001 ↗
57. Detection of pinholes in steel slabs using Gabor filter combination and morphological features10.2355/isijinternational.ISIJINT-2016-160 ↗
58. Surface Defects Classification of Hot-Rolled Steel Strips Using Multi-directional Shearlet Features10.1007/s13369-018-3329-5 ↗
59. Automatic defect detection on hot-rolled flat steel products10.1109/TIM.2012.2218677 ↗
60. Algorithm for detecting seam cracks in steel plates using a Gabor filter combination method10.1364/AO.53.004865 ↗
61. Improved contourlet transform construction and its application to surface defect recognition of metals10.1007/s11045-020-00720-5 ↗
62. Borselli, A., Colla, V., Vannucci, M., and Veroli, M. (2010, January 18–23). A fuzzy inference system applied to defect detection in flat steel production. Proceedings of the International Conference on Fuzzy Systems, Barcelona, Spain.10.1109/FUZZY.2010.5584036 ↗
63. Genetic algorithm and mathematical morphology based binarization method for strip steel defect image with non-uniform illumination10.1016/j.jvcir.2015.04.005 ↗
64. Taştimur, C., Karaköse, M., Akın, E., and Aydın, I. (2016, January 19–21). Rail defect detection with real time image processing technique. Proceedings of the 2016 IEEE 14th International Conference on Industrial Informatics (INDIN), Poitiers, France.10.1109/INDIN.2016.7819194 ↗
65. Surface defect detection method using saliency linear scanning morphology for silicon steel strip under oil pollution interference10.2355/isijinternational.54.2598 ↗
66. A simple guidance template-based defect detection method for strip steel surfaces10.1109/TII.2018.2887145 ↗
67. Zhou, S., Wu, S., Liu, H., Lu, Y., and Hu, N. (2018). Double low-rank and sparse decomposition for surface defect segmentation of steel sheet. Appl. Sci., 8.10.3390/app8091628 ↗
68. Identification of weld defects using magneto-optical imaging10.1007/s00170-019-04401-x ↗
69. Application of hidden Markov tree model to on-line detection of surface defects for steel strips10.3901/JME.2013.22.034 ↗
70. Surface defect detection via entity sparsity pursuit with intrinsic priors10.1109/TII.2019.2917522 ↗
71. Automated surface feature detection using fringe projection: An autoregressive modeling-based approach10.1016/j.optlaseng.2019.05.014 ↗
72. Surface detection of continuous casting slabs based on curvelet transform and kernel locality preserving projections10.1016/S1006-706X(13)60102-8 ↗
73. Hwang, Y.I., Seo, M.K., Oh, H.G., Choi, N., Kim, G., and Kim, K.B. (2022). Detection and classification of artificial defects on stainless steel plate for a liquefied hydrogen storage vessel using short-time fourier transform of ultrasonic guided waves and linear discriminant analysis. Appl. Sci., 12.10.3390/app12136502 ↗
74. Machine vision intelligence for product defect inspection based on deep learning and Hough transform10.1016/j.jmsy.2019.03.002 ↗
75. Visual pattern recognition by moment invariants10.1109/TIT.1962.1057692 ↗
76. Classification of defects in steel strip surface based on multiclass support vector machine10.1007/s11042-012-1248-0 ↗
77. Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm10.1016/j.neucom.2015.05.134 ↗
78. Image quality assessment based on Prewitt magnitude10.1016/j.aeue.2013.04.001 ↗
79. A deep-learning-based approach for fast and robust steel surface defects classification10.1016/j.optlaseng.2019.05.005 ↗
80. Defect detection of hot rolled steels with a new object detection framework called classification priority network10.1016/j.cie.2018.12.043 ↗
81. Konovalenko, I., Maruschak, P., Brezinová, J., Viňáš, J., and Brezina, J. (2020). Steel surface defect classification using deep residual neural network. Metals, 10.10.3390/met10060846 ↗
82. Visual inspection of steel surface defects based on domain adaptation and adaptive convolutional neural network10.1016/j.ymssp.2020.107541 ↗
83. Feng, X., Gao, X., and Luo, L. (2021). X-SDD: A new benchmark for hot rolled steel strip surface defects detection. Symmetry, 13.10.3390/sym13040706 ↗
84. Edge-Aware Multi-Level Interactive Network for Salient Object Detection of Strip Steel Surface Defects10.1109/ACCESS.2021.3124814 ↗
85. Li, Z., Wu, C., Han, Q., Hou, M., Chen, G., and Weng, T. (2022). CASI-Net: A novel and effect steel surface defect classification method based on coordinate attention and self-interaction mechanism. Mathematics, 10.10.3390/math10060963 ↗
86. A fast learning algorithm for deep belief nets10.1162/neco.2006.18.7.1527 ↗
87. Solar cells surface defects detection based on deep learning
88. X-ray inspection of TSV defects with self-organizing map network and Otsu algorithm10.1016/j.microrel.2016.10.011 ↗
89. Steel surface defect detection using a new Haar-Weibull-variance model in unsupervised manner10.1109/TIM.2017.2712838 ↗
90. An unsupervised-learning-based approach for automated defect inspection on textured surfaces10.1109/TIM.2018.2795178 ↗
91. Liu, K., Li, A., Wen, X., Chen, H., and Yang, P. (2019, January 5–7). Steel surface defect detection using GAN and one-class classifier. Proceedings of the 2019 25th International Conference on Automation and Computing (ICAC), Lancaster, UK.10.23919/IConAC.2019.8895110 ↗
92. Youkachen, S., Ruchanurucks, M., Phatrapomnant, T., and Kaneko, H. (2019, January 25–27). Defect segmentation of hot-rolled steel strip surface by using convolutional auto-encoder and conventional image processing. Proceedings of the 2019 10th International Conference of Information and Communication Technology for Embedded Systems (IC-ICTES), Bangkok, Thailand.10.1109/ICTEmSys.2019.8695928 ↗
93. Unsupervised saliency detection of rail surface defects using stereoscopic images
94. A brief introduction to weakly supervised learning10.1093/nsr/nwx106 ↗
95. Surface defect classification of steels with a new semi-supervised learning method10.1016/j.optlaseng.2019.01.011 ↗
96. Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., and Raffel, C.A. (2019). Mixmatch: A holistic approach to semi-supervised learning. Adv. Neural Inf. Process. Syst., 32.
97. Semi-supervised defect classification of steel surface based on multi-training and generative adversarial network10.1016/j.optlaseng.2019.06.020 ↗
98. Automated defect inspection system for metal surfaces based on deep learning and data augmentation10.1016/j.jmsy.2020.03.009 ↗
99. Mixed supervision for surface-defect detection: From weakly to fully supervised learning10.1016/j.compind.2021.103459 ↗
100. Segmentation-based deep-learning approach for surface-defect detection10.1007/s10845-019-01476-x ↗