Efficient Defect Classification Using Few-Shot Image Generation and Self-Attention Fused Convolution Features
Source abstract
Abstract
Although deep learning has been proven to significantly outperform most traditional methods in the classification of large-scale balanced image datasets, collecting enough samples for defect classification is extremely time-consuming and costly. In this paper, we propose a lightweight defect classification method based on few-shot image generation and self-attention fused convolution features. We constructed a 4-class dataset using welding seam images collected from a solar cell module packaging production line. To address the issue of limited defect samples, especially for classes with less than 10 images, we implemented two strategies. Geometric enhancement techniques were first used to extend the defective images. Secondly, multi-scale feature fusion Generative Adversarial Networks (GANs) were utilized to further enhance the dataset. We then performed the feature-level fusion of convolution neural networks and self-attention networks, achieving a classification accuracy of 98.19%. Our experimental results demonstrate that the proposed model performs well in small sample defect classification tasks. And, it can be effectively applied to product quality inspection tasks in industrial production lines.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Non-Destructive Testing Techniques · Welding Techniques and Residual Stresses
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
National Center for Materials Service Safety, University of Science and Technology Beijing, Beijing 100083, China
Southern Marine Science and Engineering Guangdong Laboratory, Zhuhai 519080, China
Funding
National Natural Science Foundation of China · Innovation Group Project of the Southern Marine Science and Engineering Guangdong Laboratory (Zhuhai)
References (44)
Show source-provided references
1. Hyperspectral image classification using principle component analysis and deep convolutional neural network10.1007/s12652-022-03876-z ↗
2. Mask-guided SSD for small-object detection10.1007/s10489-020-01949-0 ↗
3. Machine vision based image processing techniques for surface finish and defect inspection in a grinding process10.1016/j.matpr.2018.02.263 ↗
4. Kumar, J., Srivastava, S., Anand, R.S., Arvind, P., Bhardwaj, S., and Thakur, A. (2018, January 1–2). GLCM and ANN based approach for classification of radiographics weld images. Proceedings of the 2018 IEEE 13th International Conference on Industrial and Information Systems (ICIIS), Rupnagar, India.10.1109/ICIINFS.2018.8721421 ↗
5. Wood defect classification based on two-dimensional histogram constituted by LBP and local binary differential excitation pattern10.1109/ACCESS.2019.2945355 ↗
6. Ma, R., Deng, S., Sun, H., and Qi, Y. (2019, January 6–8). An algorithm for fabric defect detection based on adaptive canny operator. Proceedings of the 2019 International Conference on Intelligent Computing, Automation and Systems (ICICAS), Chongqing, China.10.1109/ICICAS48597.2019.00105 ↗
7. Xuan, L., and Hong, Z. (2017, January 24–26). An improved canny edge detection algorithm. Proceedings of the 2017 8th IEEE international Conference on Software Engineering and Service Science (ICSESS), Beijing, China.10.1109/ICSESS.2017.8342913 ↗
8. Local sharpness failure detection of camera module lens based on image blur assessment10.1007/s10489-022-03948-9 ↗
9. DM-RIS: Deep multimodel rail inspection system with improved MRF-GMM and CNN10.1109/TIM.2019.2909940 ↗
10. Pastor-López, I., Sanz, B., de la Puerta, J.G., and Bringas, P.G. (2019, January 4–6). Surface defect modelling using co-occurrence matrix and fast fourier transformation. Proceedings of the Hybrid Artificial Intelligent Systems: 14th International Conference, HAIS 2019, Proceedings 14, León, Spain.10.1007/978-3-030-29859-3_63 ↗
11. Using deep learning for defect classification on a small weld X-ray image dataset10.1007/s10921-020-00719-9 ↗
12. Deep learning approach for defective spot welds classification using small and class-imbalanced datasets10.1016/j.neucom.2022.01.004 ↗
13. A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks10.1016/j.autcon.2019.02.013 ↗
14. Automated defect detection in digital radiography of aerospace welds using deep learning10.1007/s40194-022-01257-w ↗
15. Deep features based on a DCNN model for classifying imbalanced weld flaw types10.1016/j.measurement.2018.09.011 ↗
16. Generative Adversarial Nets
17. Radford, A., Metz, L., and Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv.
18. Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. arXiv.
19. Odena, A., Olah, C., and Shlens, J. (2017, January 6–11). Conditional image synthesis with auxiliary classifier gans. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia.
20. Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6–11). Wasserstein generative adversarial networks. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia.
21. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.C. (2017). Improved training of wasserstein gans. Adv. Neural Inf. Process. Syst., 30.
22. Mariani, G., Scheidegger, F., Istrate, R., Bekas, C., and Malossi, C. (2018). Bagan: Data augmentation with balancing gan. arXiv.
23. Enhanced balancing GAN: Minority-class image generation10.1007/s00521-021-06163-8 ↗
24. Ojha, U., Li, Y., Lu, J., Efros, A.A., Lee, Y.J., Shechtman, E., and Zhang, R. (2021, January 20–25). Few-shot image generation via cross-domain correspondence. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01060 ↗
25. Xiao, J., Li, L., Wang, C., Zha, Z.J., and Huang, Q. (2022, January 18–24). Few shot generative model adaption via relaxed spatial structural alignment. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01092 ↗
26. Ding, G., Han, X., Wang, S., Wu, S., Jin, X., Tu, D., and Huang, Q. (2022, January 18–24). Attribute Group Editing for Reliable Few-shot Image Generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01091 ↗
27. Differentiable augmentation for data-efficient gan training
28. Yang, M., Wang, Z., Chi, Z., and Feng, W. (2022, January 23–27). WaveGAN: Frequency-Aware GAN for High-Fidelity Few-Shot Image Generation. Proceedings of the Computer Vision—ECCV 2022: 17th European Conference, Proceedings, Part XV, Tel Aviv, Israel.10.1007/978-3-031-19784-0_1 ↗
29. Liu, B., Zhu, Y., Song, K., and Elgammal, A. (2021, January 4). Towards faster and stabilized gan training for high-fidelity few-shot image synthesis. Proceedings of the International Conference on Learning Representations, Vienna, Austria.
30. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
31. Dutta, S., Basarab, A., Georgeot, B., and Kouamé, D. (2022, January 16–19). Deep Unfolding of Image Denoising by Quantum Interactive Patches. Proceedings of the 2022 IEEE International Conference on Image Processing (ICIP), Bordeaux, France.10.1109/ICIP46576.2022.9897959 ↗
32. Dutta, S., Basarab, A., Georgeot, B., and Kouamé, D. (2022). DIVA: Deep Unfolded Network from Quantum Interactive Patches for Image Restoration. arXiv.
33. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. arXiv.10.1109/ICCV48922.2021.00986 ↗
34. Lou, M., Zhou, H.Y., Yang, S., and Yu, Y. (2023). TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition. arXiv.
35. Wu, H., Xiao, B., Codella, N., Liu, M., Dai, X., Yuan, L., and Zhang, L. (2021). CvT: Introducing Convolutions to Vision Transformers. arXiv.10.1109/ICCV48922.2021.00009 ↗
36. Guo, J., Han, K., Wu, H., Tang, Y., Chen, X., Wang, Y., and Xu, C. (2022). CMT: Convolutional Neural Networks Meet Vision Transformers. arXiv.10.1109/CVPR52688.2022.01186 ↗
37. Mehta, S., and Rastegari, M. (2022). MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer. arXiv.
38. Lim, J.H., and Ye, J.C. (2017). Geometric gan. arXiv.
39. Tran, D., Ranganath, R., and Blei, D.M. (2017). Deep and hierarchical implicit models. arXiv.
40. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning10.1016/j.neunet.2017.12.012 ↗
41. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. (2017, January 4–9). GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. Proceedings of the NIPS’17, Red Hook, NY, USA.
42. Visualizing data using t-SNE
43. Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., and Pang, R. (November, January 27). Searching for MobileNetV3. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00140 ↗
44. Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. (2017, January 21–26). Aggregated Residual Transformations for Deep Neural Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.634 ↗