Research on Six-Degree-of-Freedom Refueling Robotic Arm Positioning and Docking Based on RGB-D Visual Guidance
Source abstract
Abstract
The main contribution of this paper is the proposal of a six-degree-of-freedom (6-DoF) refueling robotic arm positioning and docking technology guided by RGB-D camera visual guidance, as well as conducting in-depth research and experimental validation on the technology. We have integrated the YOLOv8 algorithm with the Perspective-n-Point (PnP) algorithm to achieve precise detection and pose estimation of the target refueling interface. The focus is on resolving the recognition and positioning challenges of a specialized refueling interface by the 6-DoF robotic arm during the automated refueling process. To capture the unique characteristics of the refueling interface, we developed a dedicated dataset for the specialized refueling connectors, ensuring the YOLO algorithm’s accurate identification of the target interfaces. Subsequently, the detected interface information is converted into precise 6-DoF pose data using the PnP algorithm. These data are used to determine the desired end-effector pose of the robotic arm. The robotic arm’s movements are controlled through a trajectory planning algorithm to complete the refueling gun docking process. An experimental setup was established in the laboratory to validate the accuracy of the visual recognition and the applicability of the robotic arm’s docking posture. The experimental results demonstrate that under general lighting conditions, the recognition accuracy of this docking interface method meets the docking requirements. Compared to traditional vision-guided methods based on OpenCV, this visual guidance algorithm exhibits better adaptability and effectively provides pose information for the robotic arm.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Aerospace Engineering and Control Systems · Robotic Path Planning Algorithms · Robotics and Sensor-Based Localization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechanical and Material Engineering, North China University of Technology, Beijing 100144, China
Funding
National Natural Science Foundation of China
References (29)
Show source-provided references
1. Harris, C., and Stephens, M. (September, January 31). A combined corner and edge detector. Proceedings of the Alvey Vision Conference, Manchester, UK.
2. Object recognition from local scale-invariant features10.1109/ICCV.1999.790410 ↗
3. Speeded-up robust features (SURF)10.1016/j.cviu.2007.09.014 ↗
4. Dalal, N., and Triggs, B. (2005, January 20–25). Histograms of Oriented Gradients for Human Detection. Proceedings of the IEEE Computer Society Conference on Computer Vision & Pattern Recognition, San Diego, CA, USA.
5. A decision-theoretic generalization of on-line learning and an application to boosting10.1006/jcss.1997.1504 ↗
6. Support-vector networks10.1007/BF00994018 ↗
7. Girshick, R. (2015, January 7–13). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
8. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
9. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020). End-to-End Object Detection with Transformers. arXiv.10.1007/978-3-030-58452-8_13 ↗
10. Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., and Dong, L. (2022, January 18–24). Swin transformer v2: Scaling up capacity and resolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01170 ↗
11. Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020, January 13–19). YOLOv4: Optimal Speed and Accuracy of Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Seattle, WA, USA.
12. Haralick, R.M., Lee, C.-N., Ottenberg, K., and Noelle, M. (1991, January 3–6). Analysis and solution of the three point perspective pose estimation problem. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Maui, HI, USA.
13. Hartley, R., and Zisserman, A. (2003). Multiple View Geometry in Computer Vision, Cambridge University Press.10.1017/CBO9780511811685 ↗
14. A Brief History and Overview of Bundle Adjustment
15. EPnP: An Accurate O(n) Solution to the PnP Problem10.1007/s11263-008-0152-6 ↗
16. Zheng, Y., Kuang, Y., Sugimoto, S., Astrom, K., and Okutomi, M. (2013, January 1–8). Revisiting the PnP Problem: A Fast, General and Optimal Solution. Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.291 ↗
17. Kneip, L., Li, H., and Seo, Y. (2014, January 6–12). UPnP: An Optimal O(n) Solution to the Absolute Pose Problem with Universal Applicability. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10590-1_9 ↗
18. A robust O(n)solution to theperspective-n-point problem10.1109/TPAMI.2012.41 ↗
19. Kendall, A., Grimes, M., and Cipolla, R. (2015, January 7–13). Posenet: A convolutional network for real-time 6-dof camera relocalization. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.336 ↗
20. Kehl, W., Manhardt, F., Tombari, F., Ilic, S., and Navab, N. (2017, January 22–29). SSD-6D: Making RGB-based 3D detection and 6d pose estimation greatagain. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.169 ↗
21. Do, T.T., Cai, M., Pham, T., and Reid, I. (2018). Deep-6DPose: Recovering 6D Object Pose from a Single RGB Image. arXiv.
22. Xiang, Y., Schmidt, T., Narayanan, V., and Fox, D. (2018, January 26–30). PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes. Proceedings of the Robotics: Science and Systems 2018, Pittsburgh, PA, USA.10.15607/RSS.2018.XIV.019 ↗
23. Rad, M., and Lepetit, V. (2017, January 22–29). BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.413 ↗
24. Tekin, B., Sinha, S.N., and Fua, P. (2018, January 18–23). Real-time seamless single shot 6d object pose prediction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00038 ↗
25. Peng, S., Liu, Y., Huang, Q., Zhou, X., and Bao, H. (2019, January 15–20). Pvnet: Pixel-wise voting network for 6dof pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00469 ↗
26. Liang, M. (2021). Study of a Binocular Vision System for Automotive Fueling Robots. [Master’s Thesis, South China University of Technology].
27. Ma, Z. (2022). Research on Fuel Tank Cap Recognition and Detection Technology Based on Binocular Vision. [Master’s Thesis, Jilin University].
28. MS-KF Fusion Algorithm for Cone Sleeve Tracking
29. Scott, G.P., Henshaw, C.G., Walker, I.D., and Willimon, B. (October, January 28). Autonomous robotic refueling of an unmanned surface vehicle in varying sea states. Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.10.1109/IROS.2015.7353591 ↗