A Defect Detection Method Based on YOLOv7 for Automated Remanufacturing
Source abstract
Abstract
Remanufacturing of mechanical parts has recently gained much attention due to the rapid development of green technologies and sustainability. Recent efforts to automate the inspection step in the remanufacturing process using artificial intelligence are noticeable. In this step, a visual inspection of the end-of-life (EOL) parts is carried out to detect defective regions for restoration. This operation relates to the object detection process, a typical computer vision task. Many researchers have adopted well-known deep-learning models for the detection of damage. A common technique in the object detection field is transfer learning, where general object detectors are adopted for specific tasks such as metal surface defect detection. One open-sourced model, YOLOv7, is known for real-time object detection, high accuracy, and optimal scaling. In this work, an investigation into the YOLOv7 behavior on various public metal surface defect datasets, including NEU-DET, NRSD, and KolektorSDD2, is conducted. A case study validation is also included to demonstrate the model’s application in an industrial setting. The tiny variant of the YOLOv7 model showed the best performance on the NEU-DET dataset with a 73.9% mAP (mean average precision) and 103 FPS (frames per second) in inference. For the NRSD dataset, the model’s base variant resulted in 88.5% for object detection and semantic segmentation inferences. In addition, the model achieved 65% accuracy when testing on the KolektorSDD2 dataset. Further, the results are studied and compared with some of the existing defect detection models. Moreover, the segmentation performance of the model was also reported.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Recycling and Waste Management Techniques · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Smart & Sustainable Manufacturing Systems Laboratory (SMART Lab), Department of Mechanical Engineering, University of Alberta, 9211 116 Street NW, Edmonton, AB T6G 2R3, Canada
Funding
Major Innovation Funds · NSERC
References (63)
Show source-provided references
1. Effective adoption of remanufacturing practices: A step towards circular economy10.1007/s13243-021-00109-y ↗
2. Production planning and control for remanufacturing: Industry practice and research needs10.1016/S0272-6963(00)00034-6 ↗
3. Developing a remanufacturing supply chain management system: A case of a successful truck engine remanufacturer in China10.1080/09537287.2016.1166282 ↗
4. Design, management and control of demanufacturing and remanufacturing systems10.1016/j.cirp.2017.05.001 ↗
5. Cloud remanufacturing: Remanufacturing enhanced through cloud technologies10.1016/j.jmsy.2022.06.003 ↗
6. A Concept for Autonomous Quality Control for Core Inspection in Remanufacturing10.1016/j.procir.2022.02.062 ↗
7. A novel deep learning-based automatic damage detection and localization method for remanufacturing/repair10.14733/cadaps.2021.1359-1372 ↗
8. Vision-based spatial damage localization method for autonomous robotic laser cladding repair processes10.1016/j.rcim.2022.102452 ↗
9. Vision-Based Damage Localization Method for an Autonomous Robotic Laser Cladding Process10.1016/j.procir.2021.11.139 ↗
10. Yang, J., Li, S., Wang, Z., Dong, H., Wang, J., and Tang, S. (2020). Using deep learning to detect defects in manufacturing: A comprehensive survey and current challenges. Materials, 13.10.3390/ma13245755 ↗
11. Visual techniques for defects detection in steel products: A comparative study10.1016/j.engfailanal.2022.106047 ↗
12. Chen, Y., Ding, Y., Zhao, F., Zhang, E., Wu, Z., and Shao, L. (2021). Surface Defect Detection Methods for Industrial Products: A Review. Appl. Sci., 11.10.3390/app11167657 ↗
13. Jin, H., Zhang, E., and Espinosa, H.D. (2023). Recent Advances and Applications of Machine Learning in Experimental Solid Mechanics: A Review. Appl. Mech. Rev., 75.10.1115/1.4062966 ↗
14. Tao, X., Zhang, D., Ma, W., Liu, X., and Xu, D. (2018). Automatic metallic surface defect detection and recognition with convolutional neural networks. Appl. Sci., 8.10.3390/app8091575 ↗
15. A Convolutional Neural Network (Cnn) for Defect Detection of Additively Manufactured Parts
16. Han, F., Liu, S., Liu, S., Zou, J., Ai, Y., and Xu, C. (2020, January 12–15). Defect detection: Defect Classification and Localization for Additive Manufacturing using Deep Learning Method. Proceedings of the 2020 IEEE 21st International Conference on Electronic Packaging Technology (ICEPT), Guangzhou, China.10.1109/ICEPT50128.2020.9202566 ↗
17. Efficient surface defect detection using self-supervised learning strategy and segmentation network10.1016/j.aei.2022.101566 ↗
18. Hot-Rolled Steel Strip Surface Inspection Based on Transfer Learning Model10.1155/2021/6637252 ↗
19. Zheng, Y., Mamledesai, H., Imam, H., and Ahmad, R. (2020, January 6–8). Deep Learning-based Automatic Damage Recognition and Spatial Localization for Remanufacturing/Repair. Proceedings of the CAD’20, Barelona, Spain.10.14733/cadconfP.2020.381-385 ↗
20. Automated rail surface crack analytics using deep data-driven models and transfer learning10.1016/j.scs.2021.102898 ↗
21. Konovalenko, I., Maruschak, P., Brezinová, J., Prentkovskis, O., and Brezina, J. (2022). Research of U-Net-Based CNN Architectures for Metal Surface Defect Detection. Machines, 10.10.3390/machines10050327 ↗
22. Litvintseva, A., Evstafev, O., and Shavetov, S. (2021, January 23–27). Real-time Steel Surface Defect Recognition Based on CNN. Proceedings of the IEEE International Conference on Automation Science and Engineering, Lyon, France.10.1109/CASE49439.2021.9551414 ↗
23. A Model for Surface Defect Detection of Industrial Products Based on Attention Augmentation
24. Dual attention deep learning network for automatic steel surface defect segmentation10.1111/mice.12792 ↗
25. Wang, J., Xu, G., Yan, F., Wang, J., and Wang, Z. (2022). Defect Transformer: An Efficient Hybrid Transformer Architecture for Surface Defect Detection. arXiv.10.1016/j.measurement.2023.112614 ↗
26. Cas-VSwin transformer: A variant swin transformer for surface-defect detection10.1016/j.compind.2022.103689 ↗
27. Alzubaidi, L., Zhang, J., Humaidi, A.J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M.A., Al-Amidie, M., and Farhan, L. (2021). Review of Deep Learning: Concepts, CNN Architectures, Challenges, Applications, Future Directions, Springer International Publishing.10.1186/s40537-021-00444-8 ↗
28. Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., and Liu, C. (2018, January 4–7). A survey on deep transfer learning. Proceedings of the Artificial Neural Networks and Machine Learning–ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece. Part III 27.10.1007/978-3-030-01424-7_27 ↗
29. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common objects in context. Proceedings of the Computer Vision—ECCV 2014: 13th European Conference, Zurich, Switzerland. Part V 13.10.1007/978-3-319-10602-1_48 ↗
30. Real-time Detection of Steel Strip Surface Defects Based on Improved YOLO Detection Network10.1016/j.ifacol.2018.09.412 ↗
31. Development of a YOLO-V3-based model for detecting defects on steel strip surface10.1016/j.measurement.2021.109454 ↗
32. Xu, Y., Zhang, K., and Wang, L. (2021). Metal surface defect detection using modified yolo. Algorithms, 14.10.3390/a14090257 ↗
33. Guo, Z., Wang, C., Yang, G., Huang, Z., and Li, G. (2022). MSFT-YOLO: Improved YOLOv5 Based on Transformer for Detecting Defects of Steel Surface. Sensors, 22.10.3390/s22093467 ↗
34. Saiz, F.A., Alfaro, G., and Barandiaran, I. (2021). An Inspection and Classification System for Automotive Component Remanufacturing Industry Based on Ensemble Learning. Information, 12.10.3390/info12120489 ↗
35. Bai, J., Wu, D., Shelley, T., Schubel, P., Twine, D., Russell, J., Zeng, X., and Zhang, J. (2024). A Comprehensive Survey on Machine Learning Driven Material Defect Detection: Challenges, Solutions, and Future Prospects. arXiv.
36. Hütten, N., Gomes, M.A., Hölken, F., Andricevic, K., Meyes, R., and Meisen, T. (2024). Deep Learning for Automated Visual Inspection in Manufacturing and Maintenance: A Survey of Open-Access Papers. Appl. Syst. Innov., 7.10.3390/asi7010011 ↗
37. Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2023, January 17–24). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00721 ↗
38. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Computer Society Conference on Computer Vision. and Pattern Recognition 2016, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
39. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, faster, stronger. Proceedings of the 30th IEEE Conference on Computer Vision. and Pattern Recognition 2017 (CVPR 2017), Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
40. Redmon, J. (2018). YOLOv3: An Incremental Improvement. arXiv.
41. Lhrg, X. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv.
42. Wang, C.-Y., Liao, H.-Y.M., Kinsley, H., Kukieła, D., and Meckel, A. (2017). Scaled-YOLOv4: Scaling Cross Stage Partial Network. arXiv.
43. Ge, Z., Liu, S., Wang, F., Li, Z., and Sun, J. (2021). YOLOX: Exceeding YOLO Series in 2021. arXiv.
44. Wang, C.-Y., Yeh, I.-H., and Liao, H.-Y.M. (2021). You Only Learn One Representation: Unified Network for Multiple Tasks. arXiv.
45. A Review of Yolo Algorithm Developments10.1016/j.procs.2022.01.135 ↗
46. Sari, E., Belbahri, M., and Nia, V.P. (2019). How Does Batch Normalization Help Binary Training?. arXiv.
47. Severstal (2024, June 20). Severstal: Steel Defect Detection. [Dataset]. Available online: https://www.kaggle.com/competitions/severstal-steel-defect-detection/data.
48. MCnet: Multiple Context Information Segmentation Network of No-Service Rail Surface Defects
49. Mixed supervision for surface-defect detection: From weakly to fully supervised learning10.1016/j.compind.2021.103459 ↗
50. Wieler, F.A.H.M., and Hahn, T. (2024, June 20). Weakly Supervised Learning for Industrial Optical Inspection [Dataset]. Available online: https://hci.iwr.uni-heidelberg.de/content/weakly-supervised-learning-industrial-optical-inspection.
51. Lv, X., Duan, F., Jiang, J.J., Fu, X., and Gan, L. (2020). Deep metallic surface defect detection: The new benchmark and detection network. Sensors, 20.10.3390/s20061562 ↗
52. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects10.1016/j.apsusc.2013.09.002 ↗
53. (2024, June 20). Ball Screw Drive Surface Defect Dataset for Classification. Institute for Production Technology. Available online: https://publikationen.bibliothek.kit.edu/1000133819.
54. Segmentation-based deep-learning approach for surface-defect detection10.1007/s10845-019-01476-x ↗
55. A Hierarchical Extractor-Based Visual Rail Surface Inspection System10.1109/JSEN.2017.2761858 ↗
56. Hu, M., and Ju, X. (2021, January 17–19). Two-stage insulator self-explosion defect detection method based on Mask R-CNN. Proceedings of the 2021 IEEE 2nd International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI), Shenyang, China.10.1109/ICHCI54629.2021.00010 ↗
57. Li, L., Jiang, Z., and Li, Y. (2021, January 25–28). Surface Defect. Detection Algorithm of Aluminum Based on Improved Faster RCNN. Proceedings of the 2021 IEEE 9th International Conference on Information, Communication and Networks (ICICN), Xi’an, China.10.1109/ICICN52636.2021.9673969 ↗
58. A survey on Image Data Augmentation for Deep Learning10.1186/s40537-019-0197-0 ↗
59. Inoue, H. (2018). Data Augmentation by Pairing Samples for Images Classification. arXiv.
60. Synthetic Data Augmentation Using GAN For Improved Automated Visual Inspection
61. Efficient Detection Model of Steel Strip Surface Defects Based on YOLO-V710.1109/ACCESS.2022.3230894 ↗
62. A pixel-level deep segmentation network for automatic defect detection10.1016/j.eswa.2022.119388 ↗
63. Learning discriminative feature representation with pixel-level supervision for forest smoke recognition10.1016/j.patcog.2023.109761 ↗