AFFNet: An Attention-Based Feature-Fused Network for Surface Defect Segmentation
Source abstract
Abstract
Recently, deep learning methods have widely been employed for surface defect segmentation in industrial production with remarkable success. Nevertheless, accurate segmentation of various types of defects is still challenging due to their irregular appearance and low contrast with the background. In light of this challenge, we propose an attention-based network with a U-shaped structure, referred to as AFFNet. In the encoder part, we present a newly designed module, Residual-RepGhost-Dblock (RRD), which focuses on the extraction of more representative features using CA attention and dilated convolution with varying expansion rates without a concomitant increase in the parameters. In the decoder part, we introduce a novel global feature attention (GFA) module to selectively fuse low-level and high-level features, suppressing distracting information such as background. Moreover, considering the imbalance of the dataset sampled from actual industrial production and the difficulty of training samples with small defects, we use the online hard sample mining (OHEM) cross-entropy loss function to improve the learning ability of hard samples. Experimental results on the NEU-seg dataset demonstrate the superiority of our method over other state-of-the-art methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Welding Techniques and Residual Stresses · Non-Destructive Testing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China
Engineering Research Center of Security Technology of Complex Network System, Ministry of Education, Northeastern University, Shenyang 110819, China
Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang 110819, China
Science and Technology on Space Physics Laboratory, Beijing 100076, China
School of Computer Science, The University of Auckland, Auckland 1010, New Zealand
Dopamine Group Ltd., Auckland 1542, New Zealand
Funding
National Natural Science Foundation of China · Fundamental Research Funds for the Central Universities
References (51)
Show source-provided references
1. Automatic visual detection system of railway surface defects with curvature filter and improved Gaussian mixture model10.1109/TIM.2018.2803830 ↗
2. Surface defect detection via entity sparsity pursuit with intrinsic priors10.1109/TII.2019.2917522 ↗
3. Generalized completed local binary patterns for time-efficient steel surface defect classification10.1109/TIM.2018.2852918 ↗
4. Deep learning10.1038/nature14539 ↗
5. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
6. NB-FCN: Real-time accurate crack detection in inspection videos using deep fully convolutional network and parametric data fusion10.1109/TIM.2019.2959292 ↗
7. Ronneberger, O., Fischer, P., and Brox, T. (2015). Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Springer. Proceedings, Part III 18.
8. Segnet: A deep convolutional encoder-decoder architecture for image segmentation10.1109/TPAMI.2016.2644615 ↗
9. Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21–26). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.660 ↗
10. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
11. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs10.1109/TPAMI.2017.2699184 ↗
12. Attention fusion network for multi-spectral semantic segmentation10.1016/j.patrec.2021.03.015 ↗
13. Yan, L., Cui, Y., Chen, Y., and Liu, D. (2021, January 6–12). Hierarchical attention fusion for geo-localization. Proceedings of the ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Virtual.10.1109/ICASSP39728.2021.9414517 ↗
14. NAS-Guided Lightweight Multiscale Attention Fusion Network for Hyperspectral Image Classification10.1109/TGRS.2021.3049377 ↗
15. MAFNet: Multi-style attention fusion network for salient object detection10.1016/j.neucom.2020.09.033 ↗
16. Invariant feature extraction method based on smoothed local binary pattern for strip steel surface defect10.2355/isijinternational.ISIJINT-2015-201 ↗
17. Automatic image thresholding using Otsu’s method and entropy weighting scheme for surface defect detection10.1007/s00500-017-2709-1 ↗
18. Classification of manufacturing defects in multicrystalline solar cells with novel feature descriptor10.1109/TIM.2019.2900961 ↗
19. Surface defect classification for hot-rolled steel strips by selectively dominant local binary patterns10.1109/ACCESS.2019.2898215 ↗
20. Steel Surface Defect Classification Based on Discriminant Manifold Regularized Local Descriptor10.1109/ACCESS.2018.2881962 ↗
21. Liu, Y., Xu, K., and Xu, J. (2019). An Improved MB-LBP Defect Recognition Approach for the Surface of Steel Plates. Appl. Sci., 9.10.3390/app9204222 ↗
22. Navarro, P.J., Fernández-Isla, C., Alcover, P.M., and Suardíaz, J. (2016). Defect detection in textures through the use of entropy as a means for automatically selecting the wavelet decomposition level. Sensors, 16.10.3390/s16081178 ↗
23. Recent advances in machine learning research for nanofluid-based heat transfer in renewable energy system10.1021/acs.energyfuels.2c01006 ↗
24. Sharma, P., and Bora, B.J. (2023). A Review of Modern Machine Learning Techniques in the Prediction of Remaining Useful Life of Lithium-Ion Batteries. Batteries, 9.10.3390/batteries9010013 ↗
25. A machine-learning-based fault diagnosis method with adaptive secondary sampling for multiphase drive systems10.1109/TPEL.2022.3153797 ↗
26. Fast defect inspection based on data-driven photometric stereo10.1109/TIM.2018.2858062 ↗
27. Tiny defect detection in high-resolution aero-engine blade images via a coarse-to-fine framework
28. Incorporating lambertian priors into surface normals measurement10.1109/TIM.2021.3096282 ↗
29. Model-driven photometric stereo for in-process inspection of non-diffuse curved surfaces10.1016/j.cirp.2019.04.013 ↗
30. MRD-net: An effective CNN-based segmentation network for surface defect detection10.1109/TIM.2022.3200361 ↗
31. Zhang, J., Ding, R., Ban, M., and Guo, T. (2022, January 22–27). FDSNeT: An Accurate Real-Time Surface Defect Segmentation Network. Proceedings of the ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9747311 ↗
32. CASDD: Automatic surface defect detection using a complementary adversarial network10.1109/JSEN.2022.3202179 ↗
33. Edge-Aware Multi-Level Interactive Network for Salient Object Detection of Strip Steel Surface Defects10.1109/ACCESS.2021.3124814 ↗
34. Li, Z., Wu, C., Han, Q., Hou, M., Chen, G., and Weng, T. (2022). CASI-Net: A Novel and Effect Steel Surface Defect Classification Method Based on Coordinate Attention and Self-Interaction Mechanism. Mathematics, 10.10.3390/math10060963 ↗
35. Nondestructive defect detection in castings by using spatial attention bilinear convolutional neural network10.1109/TII.2020.2985159 ↗
36. Dual attention deep learning network for automatic steel surface defect segmentation10.1111/mice.12792 ↗
37. Hybrid deep learning architecture for rail surface segmentation and surface defect detection10.1111/mice.12710 ↗
38. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
39. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
40. Wang, X., Girshick, R., Gupta, A., and He, K. (2018, January 18–23). Non-local neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00813 ↗
41. Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., and Lu, H. (2019, January 15–20). Dual attention network for scene segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00326 ↗
42. Chen, C., Guo, Z., Zeng, H., Xiong, P., and Dong, J. (2022). RepGhost: A Hardware-Efficient Ghost Module via Re-parameterization. arXiv.
43. Hou, Q., Zhou, D., and Feng, J. (2021, January 20–25). Coordinate attention for efficient mobile network design. Proceedings of theIEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01350 ↗
44. Zhang, Z., and Sabuncu, M. (2018, January 3–8). Generalized cross entropy loss for training deep neural networks with noisy labels. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, QC, Canada.
45. Triplet-graph reasoning network for few-shot metal generic surface defect segmentation10.1109/TIM.2021.3083561 ↗
46. PaddlePaddle: An Open-Source Deep Learning Platform from Industrial Practice
47. Bottou, L. (2010, January 22–27). Large-scale machine learning with stochastic gradient descent. Proceedings of the COMPSTAT’2010: 19th International Conference on Computational Statistics, Paris, France. Keynote, Invited and Contributed Papers 2010.10.1007/978-3-7908-2604-3_16 ↗
48. Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., and Liang, J. (2018). Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Proceedings of the 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, 20 September 2018, Springer. Proceedings 4.
49. Deep high-resolution representation learning for visual recognition10.1109/TPAMI.2020.2983686 ↗
50. Chao, P., Kao, C.Y., Ruan, Y.S., Huang, C.H., and Lin, Y.L. (November, January 27). Hardnet: A low memory traffic network. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
51. Li, X., Zhong, Z., Wu, J., Yang, Y., Lin, Z., and Liu, H. (November, January 27). Expectation-maximization attention networks for semantic segmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.