The Amalgamation of the Object Detection and Semantic Segmentation for Steel Surface Defect Detection
Source abstract
Abstract
Steel surface defect detection is challenging because it contains various atypical defects. Many studies have attempted to detect metal surface defects using deep learning and had success in applying deep learning. Despite many previous studies to solve the steel surface defect detection, it remains a difficult problem. To resolve the atypical defects problem, we introduce a hierarchical approach for the classification and detection of defects on the steel surface. The proposed approach has a hierarchical structure of the binary classifier at the first stage and the object detection and semantic segmentation algorithms at the second stage. It shows 98.6% accuracy in scratch and other types of defect classification and 77.12% mean average precision (mAP) in defect detection using the Northeastern University (NEU) surface defect detection dataset. A comparative analysis with the previous studies shows that the proposed approach achieves excellent results on the NEU dataset.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Integrated Circuits and Semiconductor Failure Analysis · Non-Destructive Testing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, Kongju National University, 1223-24 Cheonan-daero, 275 Budae-dong, Seobuk-gu, Cheonan-si 31080, Chungcheongnam-do, Korea
School of Creative Convergence, Andong National University, 1375 Gyeongdong-ro, Andong 36729, Gyeongsangbuk-do, Korea
Funding
MIST (Ministry of Science, ICT), Korea
References (51)
Show source-provided references
1. Deep learning assisted vision inspection of resistance spot welds10.1016/j.jmapro.2020.12.015 ↗
2. Development of a smart system based on STEP-NC for machine vision inspection with IoT environmental10.1007/s00170-021-08095-y ↗
3. Muresan, M.P., Cireap, D.G., and Giosan, I. (2020, January 3–5). Automatic vision inspection solution for the manufacturing process of automotive components through plastic injection molding. Proceedings of the 16th International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania.10.1109/ICCP51029.2020.9266249 ↗
4. Machine vision intelligence for product defect inspection based on deep learning and Hough transform10.1016/j.jmsy.2019.03.002 ↗
5. A machine vision algorithm for quality control inspection of gears10.1007/s00170-019-04426-2 ↗
6. A contrast adjustment thresholding method for surface defect detection based on mesoscopy10.1109/TII.2015.2417676 ↗
7. Chetverikov, D. (2000, January 3–7). Structural defects: General approach and application to textile inspection. Proceedings of the 15th International Conference on Pattern Recognition (ICPR), Barcelona, Spain.
8. Feature selection for surface defect classification of extruded aluminum profiles10.1007/s00170-015-7514-3 ↗
9. Shumin, D., Zhoufeng, L., and Chunlei, L. (2011, January 26–28). AdaBoost learning for fabric defect detection based on HOG and SVM. Proceedings of the 2011 International Conference on Multimedia Technology (ICMT), Hangzhou, China.
10. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects10.1016/j.apsusc.2013.09.002 ↗
11. Fault diagnosis of photovoltaic modules through image processing and Canny edge detection on field thermographic measurements10.1080/14786451.2013.826223 ↗
12. Rail defect detection and classification with real time image processing technique
13. Fabric defect detection using morphological filters10.1016/j.imavis.2009.03.007 ↗
14. Saliency-Based Defect Detection in Industrial Images by Using Phase Spectrum10.1109/TII.2014.2359416 ↗
15. Automated defect detection in textured surfaces using optimal elliptical Gabor filters10.1016/j.ijleo.2015.04.017 ↗
16. An Optical Surface Inspection and Automatic Classification Technique Using the Rotated Wavelet Transform10.1109/TIM.2017.2783098 ↗
17. Mandelbrot, B.B. (1982). The Fractal Geometry of Nature, WH Freeman.
18. Markov random fields and their applications
19. Hajimowlana, S.H., Muscedere, R., Jullien, G.A., and Roberts, J.W. (1998, January 9–12). 1D autoregressive modeling for defect detection in web inspection systems. Proceedings of the 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268) (MWSCAS), Notre Dame, IN, USA.
20. Lv, X., Duan, F., Jiang, J.J., Fu, X., and Gan, L. (2020). Deep Metallic Surface Defect Detection: The New Benchmark and Detection Network. Sensors, 20.10.3390/s20061562 ↗
21. Steel Surface Defect Detection using Deep Learning
22. Liu, Y., Geng, J., Su, Z., Zhang, W., and Li, J. (2018, January 4). Real-time classification of steel strip surface defects based on deep CNNs. Proceedings of the 2018 Chinese Intelligent Systems Conference (CISC), Wenzhou, China.10.1007/978-981-13-2291-4_26 ↗
23. Andrei-Alexandru, T., and Henrietta, D.E. (2020, January 21–23). Low cost defect detection using a deep convolutional neural network. Proceedings of the IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR), Cluj-Napoca, Romania.10.1109/AQTR49680.2020.9130004 ↗
24. Simonyan, K., and Zisserman, A. (2015, January 7–9). Very deep convolutional networks for large-scale image recognition. Proceedings of the International Conference of Learning Representations (ICLR), San Diego, CA, USA.
25. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features10.1109/TIM.2019.2915404 ↗
26. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
27. Li, K., Wang, X., and Ji, L. (2019, January 16–18). Application of multi-scale feature fusion and deep learning in detection of steel strip surface defect. Proceedings of the IEEE International Conference on Artificial Intelligence and Advance Manufacturing (AIAM), Dublin, Ireland.10.1109/AIAM48774.2019.00136 ↗
28. Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7–12). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Proceedings of the Advances in Neural Information Processing Systems 28 (NIPS 2015), Montréal, QC, Canada.
29. Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., and LeCun, Y. (2013). Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv.
30. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 17). SSD: Single Shot Multibox Detector. Proceedings of the Eu. Conf. on Comp. Vis. (ECCV), Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
31. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, Faster, Stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
32. Real-time detection of steel strip surface defects based on improved YOLO detection network10.1016/j.ifacol.2018.09.412 ↗
33. Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
34. Surface defect detection of steel strips based on classification priority YOLOv3-dense network10.1080/03019233.2020.1816806 ↗
35. Youkachen, S., Ruchanurucks, M., Phatrapomnant, T., and Kaneko, H. (2019, January 25–27). Defect Segmentation of Hot-rolled Steel Strip Surface by using Convolutional Auto-Encoder and Conventional Image processing. Proceedings of the 10th International Conference on Information and Communication Technology for Embedded System (IC-ICTES), Bangkok, Thailand.10.1109/ICTEmSys.2019.8695928 ↗
36. Hot-Rolled Steel Strip Surface Inspection Based on Transfer Learning Model10.1155/2021/6637252 ↗
37. Enshaei, N., Ahmad, S., and Naderkhani, F. (2020, January 8–10). Automated detection of textured-surface defects using UNet-based semantic segmentation network. Proceedings of the IEEE International Conference on Prognostics and Health Management (ICPHM), Detroit, MI, USA.10.1109/ICPHM49022.2020.9187023 ↗
38. Neven, R., and Goedemé, T. (2021). A Multi-Branch U-Net for Steel Surface Defect Type and Severity Segmentation. Metals, 11.10.3390/met11060870 ↗
39. Automatic surface defect segmentation for hot-rolled steel strip using depth-wise separable U-shape network10.1016/j.matlet.2021.130271 ↗
40. Wang, K., Wang, Y., Zhou, L., Wang, Z., and Zhang, G. (2021, January 24–26). A New Method of Surface Defect Semantic Segmentation of Steel Ball Based on Pre-Trained U-Net Model. Proceedings of the IEEE International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technlogy (CEI), Fuzhou, China.10.1109/CEI52496.2021.9574584 ↗
41. PGA-Net: Pyramid Feature Fusion and Global Context Attention Network for Automated Surface Defect Detection10.1109/TII.2019.2958826 ↗
42. Pan, Y., and Zhang, L. (2021). Dual attention deep learning network for automatic steel surface defect segmentation. Comput.-Aided Civ. Infrastruct. Eng.10.1111/mice.12792 ↗
43. TAS2-Net: Triple Attention Semantic Segmentation Network for Small Surface Defect Detection
44. Damacharla, P., Rao, M.V.A., Ringenberg, J., and Javaid, A.Y. (2021, January 19–21). IEEE TLU-Net: A Deep Learning Approach for Automatic Steel Surface Defect Detection. Proceedings of the International Conference on Applied Artificial Intelligence (ICAPAI), Halden, Norway.10.1109/ICAPAI49758.2021.9462060 ↗
45. Ali, A.A., Chramcov, B., Jasek, R., Katta, R., Krayem, S., Kadi, M., Silhavy, R., Silhavy, P., and Prokopova, Z. (2021). Detection of Steel Surface Defects Using U-Net with Pre-trained Encoder. Software Engineering Application in Informatics, Springer.10.1007/978-3-030-90318-3_18 ↗
46. Chen, T., and Guestrin, C. (2016, January 13–17). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’16), San Francisco, CA, USA.10.1145/2939672.2939785 ↗
47. Focal Loss for Dense Object Detection10.1109/TPAMI.2018.2858826 ↗
48. Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature Pyramid Networks for Object Detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
49. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Springer.10.1007/978-3-319-24574-4_28 ↗
50. A review on evaluation metrics for data classification evaluations10.5121/ijdkp.2015.5201 ↗
51. Padilla, R., Netto, S.L., and DaSilva, E.A. (2020, January 1–3). A survey on performance metrics for object-detection algorithms. Proceedings of the International Conference on Systems, Signals and Image Processing (IWSSIP), Niteroi, Brazil.10.1109/IWSSIP48289.2020.9145130 ↗