Surface Defect Detection Methods for Industrial Products: A Review
Source abstract
Abstract
The comprehensive intelligent development of the manufacturing industry puts forward new requirements for the quality inspection of industrial products. This paper summarizes the current research status of machine learning methods in surface defect detection, a key part in the quality inspection of industrial products. First, according to the use of surface features, the application of traditional machine vision surface defect detection methods in industrial product surface defect detection is summarized from three aspects: texture features, color features, and shape features. Secondly, the research status of industrial product surface defect detection based on deep learning technology in recent years is discussed from three aspects: supervised method, unsupervised method, and weak supervised method. Then, the common key problems and their solutions in industrial surface defect detection are systematically summarized; the key problems include real-time problem, small sample problem, small target problem, unbalanced sample problem. Lastly, the commonly used datasets of industrial surface defects in recent years are more comprehensively summarized, and the latest research methods on the MVTec AD dataset are compared, so as to provide some reference for the further research and development of industrial surface defect detection technology.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Infrastructure Maintenance and Monitoring · Surface Roughness and Optical Measurements
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information Science, Xi’an University of Technology, Xi’an 710048, China
Shanxi Provincial Key Laboratory of Printing and Packaging Engineering, Xi’an University of Technology, Xi’an 710048, China
Funding
National Key R&D Program of China · the Scientific Research Program funded by Shaanxi Provincial Education Department · the Key Research and Development Program of Shaanxi
References (100+)
Show source-provided references
1. Row-level algorithm to improve real-time performance of glass tube defect detection in the production phase10.1049/iet-ipr.2019.1506 ↗
2. Rasheed, A., Zafar, B., Rasheed, A., Ail, N., Sajid, M., Dar, S.H., Habib, U., Shehryar, T., and Mahmood, M.T. (2020). Fabric Defect Detection Using Computer Vision Techniques: A Comprehensive Review. Math. Probl. Eng., 2020.10.1155/2020/8189403 ↗
3. Jain, S., Seth, G., Paruthi, A., Soni, U., and Kumar, G. (2020). Synthetic data augmentation for surface defect detection and classification using deep learning. J. Int. Manufact., in press.10.1007/s10845-020-01710-x ↗
4. Adaptive segmentation algorithm for metal surface defects
5. Review of Development and Application of Defect Detection Technology
6. Deep learning with big data: State of the art and development
7. Deep Learning for Real-time Applications: A Survey
8. Visual inspection of surface defects based on lightweight reconstruction network
9. Research and development on deep learning
10. A Survey of Surface Defect Detection Methods Based on Deep Learning
11. Review on Automated Optical (Visual) Inspection and Its Applications in Defect Detection
12. Yang, J., Li, S., Wang, Z., Dong, H., Wang, J., and Tang, S. (2020). Using Deep Learning to Detect Defects in Manufacturing: A Comprehensive Survey and Current Challenges. Materials, 13.10.3390/ma13245755 ↗
13. Bergman, P., Fauser, M., Sattlegger, D., and Steger, C. (2019, January 15–20). MVTec AD—A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00982 ↗
14. A Review on Surface Defect Detection
15. Review of surface defect detection based on machine vision
16. Wood Knot Defects Recognition with Gray-Scale Histogram Features
17. Design of Automation System for Ceramic Surface Quality Control Using Fuzzy Logic Method at Balai Besar Keramik (BBK)10.1051/matecconf/201713500053 ↗
18. Liu, Y., Xu, K., and Xu, J. (2019). An Improved MB-LBP Defect Recognition Approach for the Surface of Steel Plates. Appl. Sci., 9.10.3390/app9204222 ↗
19. Yarn-Dyed Fabric Defect Detection Based on Autocorrelation Function And GLCM10.1515/aut-2015-0001 ↗
20. Defect Detection Algorithm in Steel Billets Using Morphological Top-Hat filter10.3182/20091014-3-CL-4011.00038 ↗
21. Defect detection of solar cells in electroluminescence images using Fourier image reconstruction10.1016/j.solmat.2011.12.007 ↗
22. Automatic detection of cracks in raw steel block using Gabor filter optimized by univariate dynamic encoding algorithm for searches (uDEAS)10.1016/j.ndteint.2009.01.007 ↗
23. Defect detection of IC wafer based on two-dimension wavelet transform10.1016/j.mejo.2010.01.015 ↗
24. Defect detection on patterned fabrics using texture periodicity and the coordinated clusters representation10.1177/0040517516660885 ↗
25. Mixed-state MRF detection algorithm for ink-jet printing motion texture
26. Magneto-optical imaging fractal feature detection for welding defects of high-strength steel
27. Visualized investigation of defect in cementitious materials with electrical resistance tomography10.1016/j.conbuildmat.2018.11.129 ↗
28. Song, W., Chen, T., Gu, Z., Gai, W., Huang, W., and Wang, B. (2015, January 11–13). Wood Materials Defects Detection Using Image Block Percentile Color Histogram and Eigenvector Texture Feature. Proceedings of the First International Conference on Information Sciences, Machinery, Materials and Energy, Chongqing, China.10.2991/icismme-15.2015.163 ↗
29. Prasitmeeboon, P., and Yau, H. (2019, January 2–5). Defect Detection of Particleboards by Visual Analysis and Machine Learning. Proceedings of the 2019 5th International Conference on Engineering, Applied Sciences and Technology, Luang Prabang, Laos.10.1109/ICEAST.2019.8802526 ↗
30. Research on Defect Detection Algorithm of Ceramic Tile Surface with Multi-feature Fusion
31. Influence of Hysteresis Effect on Contrast of Welding Defects Profile in Magneto-Optical Image10.1109/JSEN.2020.3009478 ↗
32. Li, Y., and Liu, M. (2018, January 12–14). Aerial Image Classification Using Color Coherence Vectors and Rotation & Uniform Invariant LBP Descriptors. Proceedings of the 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China.10.1109/IAEAC.2018.8577881 ↗
33. Kshirsagar, D.B., and Kulkarni, U.V. (2016, January 2–3). A generalized Neuro-Fuzzy Based Image Retrieval system with modified colour coherence vector and Texture element patterns. Proceedings of the 2016 IEEE International Conference on Advances in Electronics, Communication and Computer Technology (ICAECCT), New York, NY, USA.10.1109/ICAECCT.2016.7942558 ↗
34. Machine vision intelligence for product defect inspection based on deep learning and Hough transform10.1016/j.jmsy.2019.03.002 ↗
35. Research on Surface Defect Detection Method of E-TPU Midsole Based on Machine Vision10.4236/jcc.2020.811011 ↗
36. A hybrid defect detection method for compact camera lens10.1177/1687814017722949 ↗
37. Defect Detection in Electronic Surfaces Using Template-Based Fourier Image Reconstruction10.1109/TCPMT.2018.2873744 ↗
38. Detection of surface cutting defect on magnet using Fourier image reconstruction10.1007/s11771-016-0362-y ↗
39. Steel Surface Defect Detection Using a New Haar–Weibull-Variance Model in Unsupervised Manner10.1109/TIM.2017.2712838 ↗
40. Kim, M.S., Park, T., and Park, P. (2019, January 9–12). Classification of Steel Surface Defect Using Convolutional Neural Network with Few Images. Proceedings of the 12th Asian Control Conference (ASCC), Kitakyusyu International Conference Center, Fukuoka, Japan.
41. Unknown defect detection for printed circuit board based on multi-scale deep similarity measure method
42. Luan, C., Cui, R., Sun, L., and Lin, Z. (2020, January 25–28). A Siamese Network Utilizing Image Structural Differences for Cross-Category Defect Detection. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.10.1109/ICIP40778.2020.9191128 ↗
43. Liang, Q., Zhu, W., Sun, W., Yu, Z., Wang, Y., and Zhang, D. (2019). In-line inspection solution for codes on complex backgrounds for the plastic container industry. Measurement, 148.10.1016/j.measurement.2019.106965 ↗
44. Automatic Metallic Surface Defect Detection using ShuffleDefectNet
45. Tao, X., Zhang, D., Wang, Z., Liu, X., Zhang, H., and Xu, D. (2020). Detection of Power Line Insulator Defects Using Aerial Images Analyzed with Convolutional Neural Networks. IEEE Transactions on Systems, Man, and Cybernetics Systems, IEEE.10.1109/TSMC.2018.2871750 ↗
46. Detection of PCB Surface Defects with Improved Faster-RCNN and Feature Pyramid Network10.1109/ACCESS.2020.3001349 ↗
47. Ren, J., and Huang, X. (2020, January 17–19). Defect Detection Using Combined Deep Autoencoder and Classifier for Small Sample Size. Proceedings of the 2020 IEEE 6th International Conference on Control Science and Systems Engineering (ICCSSE), Beijing, China.10.1109/ICCSSE50399.2020.9171953 ↗
48. Deep Learning-Based System for Automatic Recognition and Diagnosis of Electrical Insulator Strings10.1109/ACCESS.2019.2931144 ↗
49. Balzategui, J., Eciolaza, L., and Arana-Arexolaleiba, N. (2020, January 12–15). Defect detection on Polycrystalline solar cells using Electroluminescence and Fully Convolutional Neural Networks. Proceedings of the 2020 IEEE/SICE International Symposium on System Integration (SII), Honolulu, HI, USA.10.1109/SII46433.2020.9026211 ↗
50. Faster multi-defect detection system in shield tunnel using combination of FCN and faster RCNN10.1177/1369433219849829 ↗
51. Surface Defect Detection Using Image Pyramid10.1109/JSEN.2020.2977366 ↗
52. Mehta, S., Azad, A.P., Chemmengath, S.A., Raykar, V., and Kalyanaraman, S. (2018, January 12–15). DeepSolarEye: Power Loss Prediction and Weakly Supervised Soiling Localization via Fully Convolutional Networks for Solar Panels. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.10.1109/WACV.2018.00043 ↗
53. Dike, H.U., Zhou, Y., Deveerasetty, K.K., and Wu, Q. (2018, January 25–27). Unsupervised Learning Based on Artificial Neural Network: A Review. Proceedings of the 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS), Shenzhen, China.10.1109/CBS.2018.8612259 ↗
54. Cohen, N., and Hoshen, Y. (2020). Sub-Image Anomaly Detection with Deep Pyramid Correspondences. arXiv.
55. Defard, T., Setkov, A., Loesch, A., and Audigier, R. (2020). PaDiM: A Patch Distribution Modeling Framework for Anomaly Detection and Localization. arXiv.10.1007/978-3-030-68799-1_35 ↗
56. Roth, K., Pemula, L., Zepeda, J., Schölkopf, B., Brox, T., and Gehler, P. (2021). Towards Total Recall in Industrial Anomaly Detection. arXiv.10.1109/CVPR52688.2022.01392 ↗
57. Bergmann, P., Löwe, S., Fauser, M., Sattlegger, D., and Steger, C. (2019, January 25–27). Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders. Proceedings of the International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP), SCITEPRESS, Prague, Czech.10.5220/0007364500002108 ↗
58. Pirnay, J., and Chai, K. (2021). Inpainting Transformer for Anomaly Detection. arXiv.10.1007/978-3-031-06430-2_33 ↗
59. Haselmann, M., Gruber, D.P., and Tabatabai, P. (2018, January 17–20). Anomaly Detection using Deep Learning based Image Completion. Proceedings of the 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA.10.1109/ICMLA.2018.00201 ↗
60. Multiscale Feature-Clustering-Based Fully Convolutional Autoencoder for Fast Accurate Visual Inspection of Texture Surface Defects10.1109/TASE.2018.2886031 ↗
61. An Unsupervised-Learning-Based Approach for Automated Defect Inspection on Textured Surfaces10.1109/TIM.2018.2795178 ↗
62. Ke, M., Lin, C., and Huang, Q. (2018, January 11–13). Anomaly detection of Logo images in the mobile phone using convolutional autoencoder. Proceedings of the 2017 4th International Conference on Systems and Informatics (ICSAI), Hangzhou, China.10.1109/ICSAI.2017.8248461 ↗
63. Masci, J., Meier, U., Cireşan, D., and Schmidhuber, J. (2011, January 14–17). Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction. Proceedings of the 21st International Conference on Artificial Neural Networks (ICANN), Espoo, Finland.10.1007/978-3-642-21735-7_7 ↗
64. Unsupervised fabric defect detection based on a deep convolutional generative adversarial network10.1177/0040517519862880 ↗
65. Liu, K., Li, A., Wen, X., Chen, H., and Yang, P. (2019, January 5–7). Steel Surface Defect Detection Using GAN and One-Class Classifier. Proceedings of the 2019 25th International Conference on Automation and Computing (ICAC), Lancaster, UK.10.23919/IConAC.2019.8895110 ↗
66. Lai, Y.T.K., and Hu, J.S. (2018, January 7–10). A Texture Generation Approach for Detection of Novel Surface Defects. Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Miyazaki, Japan.10.1109/SMC.2018.00736 ↗
67. Zhai, W., Zhu, J., Cao, Y., and Wang, Z. (2018, January 15–20). A Generative Adversarial Network Based Framework for Unsupervised Visual Surface Inspection. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462364 ↗
68. Multistage GAN for Fabric Defect Detection10.1109/TIP.2019.2959741 ↗
69. Solar Cells Surface Defects Detection Based on Deep Learning
70. Zheng, H., Yang, Y., Sun, X., and Wen, C. (2018, January 15–18). Nondestructive Detection of Anchorage Quality of Rock Bolt Based On DS-DBN-SVM. Proceedings of the 2018 International Conference on Machine Learning and Cybernetics (ICMLC), Chengdu, China.10.1109/ICMLC.2018.8526998 ↗
71. Wood inspection with non-supervised clustering10.1007/s00138-002-0084-z ↗
72. X-ray inspection of TSV defects with self-organizing map network and Otsu algorithm10.1016/j.microrel.2016.10.011 ↗
73. A Generic Semi-Supervised Deep Learning-Based Approach for Automated Surface Inspection10.1109/ACCESS.2020.3003588 ↗
74. Surface defect classification of steels with a new semi-supervised learning method10.1016/j.optlaseng.2019.01.011 ↗
75. Semi-supervised defect classification of steel surface based on multi-training and generative adversarial network10.1016/j.optlaseng.2019.06.020 ↗
76. Dai, W., Mujeeb, A., Erdt, M., and Sourin, A. (2020). Soldering defect detection in automatic optical inspection. Adv. Eng. Inform., 43.10.1016/j.aei.2019.101004 ↗
77. Mayr, M., Hoffmann, M., Maier, A., and Christlein, V. (2019, January 22–25). Weakly Supervised Segmentation of Cracks on Solar Cells Using Normalized Lp Norm. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8803116 ↗
78. A Weakly Supervised Surface Defect Detection Based on Convolutional Neural Network10.1109/ACCESS.2020.2977821 ↗
79. Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding10.1016/j.jmapro.2019.06.023 ↗
80. Xia, B., Cao, J., and Wang, C. (2019, January 22–24). SSIM-NET: Real-Time PCB Defect Detection Based on SSIM and MobileNet-V3. Proceedings of the 2019 2nd World Conference on Mechanical Engineering and Intelligent Manufacturing (WCMEIM), Shanghai, China.10.1109/WCMEIM48965.2019.00159 ↗
81. Lv, Y., Ma, L., and Jiang, H. (2019, January 19–21). A Mobile Phone Screen Cover Glass Defect Detection MODEL Based on Small Samples Learning. Proceedings of the 2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP), Wuxi, China.10.1109/SIPROCESS.2019.8868737 ↗
82. Pixel-Wise Defect Detection by CNNs without Manually Labeled Training Data10.1080/08839514.2019.1583862 ↗
83. Unsupervised-Learning-Based Feature-Level Fusion Method for Mura Defect Recognition10.1109/TSM.2017.2648856 ↗
84. Zyout, I., and Oatawneh, A. (April, January 4). Detection of PV Solar Panel Surface Defects using Transfer Learning of the Deep Convolutional Neural Networks. Proceedings of the 2020 Advances in Science and Engineering Technology International Conferences (ASET), Dubai, United Arab Emirates.10.1109/ASET48392.2020.9118382 ↗
85. Şeker, A. (2018, January 28–30). Evaluation of Fabric Defect Detection Based on Transfer Learning with Pre-trained AlexNet. Proceedings of the 2018 International Conference on Artificial Intelligence and Data Processing (IDAP), Malatya, Turkey.10.1109/IDAP.2018.8620888 ↗
86. Zhu, C., Zhou, W., Yu, H., and Xiao, S. (2019, January 26–28). Defect Detection of Emulsion Pump Body Based on Improved Convolutional Neural Network. Proceedings of the c2019 International Conference on Advanced Mechatronic Systems (ICAMechS), Kusatsu, Japan.10.1109/ICAMechS.2019.8861641 ↗
87. Volkau, I., Mujeeb, A., Dai, W., Marius, E., and Alexei, S. (2019, January 2–4). Detection Defect in Printed Circuit Boards using Unsupervised Feature Extraction Upon Transfer Learning. Proceedings of the 2019 International Conference on Cyberworlds (CW), Kyoto, Japan.10.1109/CW.2019.00025 ↗
88. Detection and Evaluation Method of Transmission Line Defects Based on Deep Learning10.1109/ACCESS.2020.2974798 ↗
89. Zeng, W., You, Z., Huang, M., Kong, Z., Yu, Y., and Le, X. (2019, January 14–19). Steel Sheet Defect Detection Based on Deep Learning Method. Proceedings of the 2019 Tenth International Conference on Intelligent Control and Information Processing (ICICIP), Marrakesh, Morocco.10.1109/ICICIP47338.2019.9012199 ↗
90. A Fully Convolutional Neural Network for Wood Defect Location and Identification10.1109/ACCESS.2019.2937461 ↗
91. Yang, D. (2019, January 22–24). Classification of Fabric Defects Based on Deep Adaptive Transfer Learning. Proceedings of the 2019 Chinese Automation Congress (CAC), Hangzhou, China.
92. Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., and Langs, G. (2017). Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery. International Conference on Information Processing in Medical Imaging, Springer.10.1007/978-3-319-59050-9_12 ↗
93. f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks10.1016/j.media.2019.01.010 ↗
94. Akcay, S., Atapour-Abarghouei, A., and Breckon, T.P. (2018). GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training. Asian Conference on Computer Vision, Springer.
95. Jeong, J., Park, H., and Kwak, N. (2017). Enhancement of SSD by concatenating feature maps for object detection. arXiv.10.5244/C.31.76 ↗
96. Cai, Z., and Vasconcelos, N. (2018, January 18–23). Cascade R-CNN: Delving into High Quality Object Detection. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00644 ↗
97. Affinity and class probability-based fuzzy support vector machine for imbalanced data sets10.1016/j.neunet.2019.10.016 ↗
98. MFC-GAN: Class-imbalanced dataset classification using Multiple Fake Class Generative Adversarial Network10.1016/j.neucom.2019.06.043 ↗
99. Li, M., Xiong, A., Wang, L., Deng, S., and Ye, J. (2020). ACO Resampling: Enhancing the performance of oversampling methods for class imbalance classification. Knowl. Based Syst., 196.10.1016/j.knosys.2020.105818 ↗
100. On the relative value of data resampling approaches for software defect prediction10.1007/s10664-018-9633-6 ↗