Real-Time Detection of Insulator Defects with Channel Pruning and Channel Distillation
Source abstract
Abstract
Insulators are essential for electrical insulation and structural support in transmission lines. With the advancement of deep learning, object detection algorithms have become primary tools for detecting insulator defects. However, challenges such as low detection accuracy for small targets, weak feature map representation, the insufficient extraction of key information, and a lack of comprehensive datasets persist. This paper introduces OD (Omni-dimensional dynamic)-YOLOV7-tiny, an enhanced insulator defect detection method. We replace the YOLOv7-tiny backbone with FasterNet and optimize the convolution structure using PConv, improving spatial feature extraction efficiency and operational speed. Additionally, we incorporate the OD (Omni-dimensional dynamic)-SlimNeck feature fusion module and a decoupled detection head to enhance accuracy. For deployment on edge devices, channel pruning and channel-wise distillation are applied, significantly reducing model parameters while maintaining high accuracy. Experimental results show that the improved model reduces parameters by 53% and increases accuracy and mean average precision (mAP) by 3.9% and 2.2%, respectively. These enhancements confirm the effectiveness of our lightweight model for insulator defect detection on edge devices.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Industrial Vision Systems and Defect Detection · Image Enhancement Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronic Information, Central South University, Changsha 410082, China
School of Automation, Central South University, Changsha 410082, China
Funding
National Natural Science Foundation of China · Major Scientific Instrument Development Project of National Natural Science Foundation of China
References (42)
Show source-provided references
1. A review on the application of deep learning target detection algorithm in overhead transmission line insulator defect detection
2. Artificial intelligence image recognition method based on convolutional neural network algorithm10.1109/ACCESS.2020.3006097 ↗
3. Wang, Y., Li, Q., Liu, Y., and Wang, C. (2023, January 12–14). Insulator defect detection based on improved YOLOv5 algorithm. Proceedings of the 2023 IEEE 12th Data Driven Control and Learning Systems Conference (DDCLS), Xiangtan, China.10.1109/DDCLS58216.2023.10166190 ↗
4. Wireless communications with unmanned aerial vehicles: Opportunities and challenges10.1109/MCOM.2016.7470933 ↗
5. A light-weight defect detection model for capacitor appearance based on the Yolov510.1016/j.measurement.2024.114717 ↗
6. Object detection in 20 years: A survey10.1109/JPROC.2023.3238524 ↗
7. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst., 25.
8. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
9. Girshick, R. (2015, January 7–13). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Washington, DC, USA.10.1109/ICCV.2015.169 ↗
10. Faster R-CNN: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
11. Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
12. Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.
13. Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2023, January 17–27). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00721 ↗
14. Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
15. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (1–July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
16. Han, S., Pool, J., Tran, J., and Dally, W. (2015). Learning both weights and connections for efficient neural network. Adv. Neural Inf. Process. Syst., 28.
17. Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H.P. (2016). Pruning Filters for Efficient ConvNets. arXiv.
18. Molchanov, P., Tyree, S., Karras, T., Aila, T., and Kautz, J. (2016). Pruning convolutional neural networks for resource efficient inference. arXiv.
19. Ye, J., Lu, X., Lin, Z., and Wang, J.Z. (2018). Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers. arXiv.
20. Huang, Z., and Wang, N. (2018, January 8–14). Data-driven sparse structure selection for deep neural networks. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01270-0_19 ↗
21. Chen, J., Kao, S.-h., He, H., Zhuo, W., Wen, S., Lee, C.-H., and Chan, S.-H.G. (2023, January 17–24). Run, Don’t walk: Chasing higher FLOPS for faster neural networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01157 ↗
22. Hua, B.-S., Tran, M.-K., and Yeung, S.-K. (2018, January 18–23). Pointwise convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00109 ↗
23. Nair, V., and Hinton, G.E. (2010, January 21–24). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel.
24. Li, H., Li, J., Wei, H., Liu, Z., Zhan, Z., and Ren, Q. (2022). Slim-neck by GSConv: A better design paradigm of detector architectures for autonomous vehicles. arXiv.
25. Deep convolutional neural networks for image classification: A comprehensive review10.1162/neco_a_00990 ↗
26. Li, C., Zhou, A., and Yao, A. (2022). Omni-dimensional dynamic convolution. arXiv.
27. Zhuang, J., Qin, Z., Yu, H., and Chen, X. (2023). Task-specific context decoupling for object detection. arXiv.
28. Ge, Z., Liu, S., Wang, F., Li, Z., and Sun, J. (2021). Yolox: Exceeding yolo series in 2021. arXiv.
29. Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., and Zhang, C. (2017, January 22–29). Learning efficient convolutional networks through network slimming. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.298 ↗
30. Ioffe, S., and Szegedy, C. (2015, January 6–11). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, Lille, France.
31. Yang, Y., Qiu, J., Song, M., Tao, D., and Wang, X. (2020, January 13–19). Distilling knowledge from graph convolutional networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00710 ↗
32. Zhou, Z., Zhuge, C., Guan, X., and Liu, W. (2020). Channel distillation: Channel-wise attention for knowledge distillation. arXiv.
33. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
34. Simon, M., and Rodner, E. (2015, January 7–13). Neural activation constellations: Unsupervised part model discovery with convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.136 ↗
35. Fitnets: Hints for thin deep nets
36. Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv.
37. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2017, January 4–9). Automatic differentiation in pytorch. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
38. Zhang, Z. (2018, January 4–6). Improved adam optimizer for deep neural networks. Proceedings of the 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), Banff, AB, Canada.10.1109/IWQoS.2018.8624183 ↗
39. Henderson, P., and Ferrari, V. (2016, January 20–24). End-to-end training of object class detectors for mean average precision. Proceedings of the Computer Vision–ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan. Revised Selected Papers, Part V 13.
40. Srinivas, S., Subramanya, A., and Venkatesh Babu, R. (2017, January 21–26). Training sparse neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.61 ↗
41. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. Proceedings, Part I 14.10.1007/978-3-319-46448-0_2 ↗
42. Tang, S., He, F., Huang, X., and Yang, J. (2019). Online PCB defect detector on a new PCB defect dataset. arXiv.