A Fitting Recognition Approach Combining Depth-Attention YOLOv5 and Prior Synthetic Dataset
Source abstract
Abstract
To address power transmission lines (PTLs) traveling through complex environments leading to misdetections and omissions in fitting recognition using cameras, we propose a fitting recognition approach combining depth-attention YOLOv5 and prior synthetic dataset to improve the validity of fitting recognition. First, datasets with inspection features are automatically synthesized based on prior series data, achieving better results with a smaller data volume for the deep learning model and reducing the cost of obtaining fitting datasets. Next, a unique data collection mode is proposed using a developed flying-walking power transmission line inspection robot (FPTLIR) as the acquisition platform. The obtained image data in this collection mode has obvious time-space, stability, and depth difference, fusing the two data types in the deep learning model to improve the accuracy. Finally, a depth-attention mechanism is proposed to change the attention on the images with depth information, reducing the probability of model misdetection and omission. Test field experiments results show that compared with YOLOv5, the mAP5095 (mean average precision on step size 0.05 for thresholds from 0.5 to 0.95) of our depth-attention YOLOv5 model for fitting is 68.1%, the recall is 98.3%, and the precision is 98.3%. Among them, AP, recall, and precision increased by 5.2%, 4.8%, and 4.1%, respectively. Test field experiments verify the feasibility of the depth-attention YOLOv5. Line field experiments results show that the mAP5095 of our depth-attention YOLOv5 model for fittings is 64.6%, and the mAPs of each class are improved compared with other attention mechanisms. The inference speed of depth-attention YOLOv5 is 3 ms slower than the standard YOLOv5 model and 10 ms to 15 ms faster than other attention mechanisms, verifying the validity of the depth-attention YOLOv5. The proposed approach improves the accuracy of the fitting recognition on PTLs, providing a recognition and localization basis for the automation and intelligence of inspection robots.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Power Line Inspection Robots · Robotics and Sensor-Based Localization · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Mechanical and Electrical Engineering, Shihezi University, Shihezi 832003, China
Funding
the National Natural Science Foundation of China · the Financial Science and Technology Program of the XPCC · the High-level Talent Project of Shihezi University
References (53)
Show source-provided references
1. Luo, Y., Yu, X., Yang, D., and Zhou, B. (2022). A survey of intelligent transmission line inspection based on unmanned aerial vehicle. Artif. Intell. Rev., in press.10.1007/s10462-022-10189-2 ↗
2. A Review on State-of-the-Art Power Line Inspection Techniques10.1109/TIM.2020.3031194 ↗
3. Fan, F., Wu, G., Wang, M., Cao, Q., and Yang, S. (2018). Multi-Robot Cyber Physical System for Sensing Environmental Variables of Transmission Line. Sensors, 18.10.3390/s18093146 ↗
4. Xinyan, Q., Gongping, W., and Jin, L. (2018). Detecting Inspection Objects of Power Line from Cable Inspection Robot LiDAR Data. Sensors, 18.10.3390/s18041284 ↗
5. Qin, X., Wu, G., Lei, J., Fan, F., Ye, X., and Mei, Q. (2018). A novel method of autonomous inspection for transmission line based on cable inspection robot lidar data. Sensors, 18.10.3390/s18020596 ↗
6. Bian, J., Hui, X., Zhao, X., and Tan, M. (2018, January 1–5). A novel monocular-based navigation approach for UAV autonomous transmission-line inspection. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8593926 ↗
7. Bian, J., Hui, X., Zhao, X., and Tan, M. (2018, January 12–15). A point-line-based SLAM framework for UAV close proximity transmission tower inspection. Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia.10.1109/ROBIO.2018.8664716 ↗
8. Power transmission line inspection robots: A review, trends and challenges for future research10.1016/j.ijepes.2020.105862 ↗
9. Texture-and-shape based active contour model for insulator segmentation10.1109/ACCESS.2019.2922257 ↗
10. Identification method of railway insulator based on edge features10.1088/1757-899X/394/3/032023 ↗
11. Insulator fault detection based on spatial morphological features of aerial images10.1109/ACCESS.2018.2846293 ↗
12. High voltage outdoor insulator surface condition evaluation using aerial insulator images10.1049/hve.2019.0079 ↗
13. Insulator detection in aerial images for transmission line inspection using single shot multibox detector10.1109/ACCESS.2019.2891123 ↗
14. Infrared evaluation classification method for deteriorated insulator based on Bayesian algorithm10.1088/1742-6596/1314/1/012080 ↗
15. An intelligent recognition system for insulator string defects based on dimension correction and optimized faster R-CNN10.1007/s00202-020-01099-z ↗
16. Ayoub, N., and Schneider-Kamp, P. (2021). Real-Time On-Board Deep Learning Fault Detection for Autonomous UAV Inspections. Electronics, 10.10.3390/electronics10091091 ↗
17. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
18. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.169 ↗
19. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks10.1109/TPAMI.2016.2577031 ↗
20. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
21. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition10.1109/TPAMI.2015.2389824 ↗
22. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
23. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
24. Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
25. Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.
26. Ge, Z., Liu, S., Wang, F., Li, Z., and Sun, J. (2021). Yolox: Exceeding yolo series in 2021. arXiv.
27. Xu, S., Wang, X., Lv, W., Chang, Q., Cui, C., Deng, K., Wang, G., Dang, Q., Wei, S., and Du, Y. (2022). PP-YOLOE: An evolved version of YOLO. arXiv.
28. Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2022). YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv.10.1109/CVPR52729.2023.00721 ↗
29. Rahman, E.U., Zhang, Y., Ahmad, S., Ahmad, H.I., and Jobaer, S. (2021). Autonomous vision-based primary distribution systems porcelain insulators inspection using UAVs. Sensors, 21.10.3390/s21030974 ↗
30. Detection of Power Line Insulator Defects Using Aerial Images Analyzed With Convolutional Neural Networks10.1109/TSMC.2018.2871750 ↗
31. Zhang, J., Qin, X., Lei, J., Jia, B., Li, B., Li, Z., Li, H., Zeng, Y., and Song, J. (2022). A Novel Auto-Synthesis Dataset Approach for Fitting Recognition Using Prior Series Data. Sensors, 22.10.3390/s22124364 ↗
32. Measurement of Power Line Sagging Using Sensor Data of a Power Line Inspection Robot10.1109/ACCESS.2020.2998154 ↗
33. Gulzar, M.A., Kumar, K., Javed, M.A., and Sharif, M. (2018, January 22–23). High-voltage transmission line inspection robot. Proceedings of the 2018 International Conference on Engineering and Emerging Technologies (ICEET), Lahore, Pakistan.10.1109/ICEET1.2018.8338632 ↗
34. Richard, P.L., Pouliot, N., Morin, F., Lepage, M., Hamelin, P., Lagac, M., Sartor, A., Lambert, G., and Montambault, S. (2019, January 20–24). LineRanger: Analysis and Field Testing of an Innovative Robot for Efficient Assessment of Bundled High-Voltage Powerlines. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8794397 ↗
35. POLIBOT–power lines inspection robot10.1108/IR-08-2016-0217 ↗
36. Mirallès, F., Hamelin, P., Lambert, G., Lavoie, S., Pouliot, N., Montfrond, M., and Montambault, S. (2018, January 21–25). LineDrone Technology: Landing an Unmanned Aerial Vehicle on a Power Line. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.10.1109/ICRA.2018.8461250 ↗
37. Automatic Obstacle-Crossing Planning for a Transmission Line Inspection Robot Based on Multisensor Fusion10.1109/ACCESS.2022.3183125 ↗
38. A novel flying–walking power line inspection robot and stability analysis hanging on the line under wind loads10.5194/ms-13-257-2022 ↗
39. Wang, H., Li, E., Yang, G., and Guo, R. (2019, January 4–7). Design of an Inspection Robot System with Hybrid Operation Modes for Power Transmission Lines. Proceedings of the 2019 IEEE International Conference on Mechatronics and Automation (ICMA), Tianjin, China.10.1109/ICMA.2019.8816540 ↗
40. Chang, W., Yang, G., Yu, J., Liang, Z., Cheng, L., and Zhou, C. (2017, January 24–28). Development of a power line inspection robot with hybrid operation modes. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.10.1109/IROS.2017.8202263 ↗
41. Jocher, G. (2022, August 08). Yolov5. Available online: https://github.com/ultralytics/yolov5.
42. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
43. Yuan, Y., Huang, L., Guo, J., Zhang, C., Chen, X., and Wang, J. (2018). Ocnet: Object context network for scene parsing. arXiv.
44. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23–28). End-to-end object detection with transformers. Proceedings of the Computer Vision(ECCV), Glasgow, UK.10.1007/978-3-030-58452-8_13 ↗
45. Squeeze-and-excitation networks10.1109/TPAMI.2019.2913372 ↗
46. Qin, Z., Zhang, P., Wu, F., and Li, X. (2021, January 10–17). Fcanet: Frequency channel attention networks. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00082 ↗
47. Hou, Q., Zhou, D., and Feng, J. (2021, January 20–25). Coordinate attention for efficient mobile network design. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01350 ↗
48. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2020, January 13–19). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01155 ↗
49. Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
50. Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., and Lu, H. (2019, January 15–20). Dual attention network for scene segmentation. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00326 ↗
51. Yang, L., Zhang, R.-Y., Li, L., and Xie, X. (2021, January 18–24). Simam: A simple, parameter-free attention module for convolutional neural networks. Proceedings of the International Conference on Machine Learning, Virtual.
52. Zhang, J. (2022, August 08). Fitting Synthetic Dataset (Insulator). Available online: https://github.com/zjlanthe/synthetic_data_insulator_blender.
53. Attention mechanisms in computer vision: A survey10.1007/s41095-022-0271-y ↗