Vision-Based Robotic Arm Control Algorithm Using Deep Reinforcement Learning for Autonomous Objects Grasping
Source abstract
Abstract
While working side-by-side, humans and robots complete each other nowadays, and we may say that they work hand in hand. This study aims to evolve the grasping task by reaching the intended object based on deep reinforcement learning. Thereby, in this paper, we propose a deep deterministic policy gradient approach that can be applied to a numerous-degrees-of-freedom robotic arm towards autonomous objects grasping according to their classification and a given task. In this study, this approach is realized by a five-degrees-of-freedom robotic arm that reaches the targeted object using the inverse kinematics method. You Only Look Once v5 is employed for object detection, and backward projection is used to detect the three-dimensional position of the target. After computing the angles of the joints at the detected position by inverse kinematics, the robot’s arm is moved towards the target object’s emplacement thanks to the algorithm. Our approach provides a neural inverse kinematics solution that increases overall performance, and its simulation results reveal its advantages compared to the traditional one. The robot’s end grip joint can reach the targeted location by calculating the angle of every joint with an acceptable range of error. However, the accuracy of the angle and the posture are satisfied. Experiments reveal the performance of our proposal compared to the state-of-the-art approaches in vision-based grasp tasks. This is a new approach to grasp an object by referring to inverse kinematics. This method is not only easier than the standard one but is also more meaningful for multi-degrees of freedom robots.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robot Manipulation and Learning · Soft Robotics and Applications · Reinforcement Learning in Robotics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Engineering Unit, Euromed Research Center, Euro-Mediterranean University, Fes 51, Morocco
Electrical Engineering Department, Hassania School of Public Labors, Casablanca 8108, Morocco
Applied Sciences Department, University of Quebec, Chicoutimi, QC G7H 2B1, Canada
References (31)
Show source-provided references
1. Danielczuk, M., Mahler, J., Correa, C., and Goldberg, K. (2018, January 20–24). Linear Push Policies to Increase Grasp Access for Robot Bin Picking. Proceedings of the IEEE 14th International Conference on Automation Science and Engineering (CASE), Munich, Germany.10.1109/COASE.2018.8560406 ↗
2. A Software Architecture for Service Robots Manipulating Objects in Human Environments10.1109/ACCESS.2020.3003991 ↗
3. Phaniteja, S., Dewangan, P., Guhan, P., Sarkar, A., and Krishna, K.M. (2017, January 5–8). A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots. Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO), Macau, Macao.10.1109/ROBIO.2017.8324682 ↗
4. Vincze, M. (2020). Learn, Detect, and Grasp Objects in Real-World Settings, Springer.10.1007/s00502-020-00817-6 ↗
5. Morrison, D., Corke, P., and Leitner, J. (2019). Learning robust, real-time, reactive robotic grasping. Int. J. Robot. Res., 19.
6. Du, G., Wang, K., Lian, S., and Zhao, K. (2020). Vision-based Robotic Grasping From Object Localization, Object Pose Estimation to Grasp Estimation for Parallel Grippers: A Review. arXiv.10.1007/s10462-020-09888-5 ↗
7. Xiao, T., Jang, E., Kalashnikov, D., Levine, S., Ibarz, J., Hausman, K., and Herzog, A. (2020). Thinking While Moving: Deep Reinforcement Learning with Concurrent Control. arXiv.
8. Hand-arm autonomous grasping: Synergistic motions to enhance the learning process10.1007/s11370-018-0262-0 ↗
9. Von Oehsen, T., Fabisch, A., Kumar, S., and Kirchner, E.F. (2020). Comparison of Distal Teacher Learning with Numerical and Analytical Methods to Solve Inverse Kinematics for Rigid-Body Mechanisms. arXiv.
10. A review On reinforcement learning: Introduction and applications in industrial process control10.1016/j.compchemeng.2020.106886 ↗
11. Nguyen, H. (2019, January 25–27). Review of Deep Reinforcement Learning for Robot Manipulation. Proceedings of the 2019 Third IEEE International Conference on Robotic Computing (IRC), Naples, Italy.10.1109/IRC.2019.00120 ↗
12. Quillen, D., Jang, E., Nachum, O., Finn, C., Ibarz, J., and Levine, S. (2018, January 21–25). Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methods. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.10.1109/ICRA.2018.8461039 ↗
13. Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges and Recommendations10.1109/ACCESS.2020.3027923 ↗
14. Aitygulov, E.E. (2019, January 4–7). The Use of Reinforcement Learning in the Task of Moving Objects with the Robotic Arm. Proceedings of the Artificial Intelligence 5th RAAI Summer School, Dolgoprudny, Russia.10.1007/978-3-030-33274-7_7 ↗
15. Franceschetti, A., Tosello, E., Castaman, N., and Ghidoni, S. (2020). Robotic Arm Control and Task Training through Deep Reinforcement Learning. arXiv.
16. Guo, Z., and Huang, J. (2019, January 26–28). A Reinforcement Learning Approach for Inverse Kinematics of Arm Robot. Proceedings of the 2019 The 4th International Conference on Robotics, Control and Automation, Guangzhou, China.10.1145/3351180.3351199 ↗
17. Joshi, S., Kumra, S., and Sahin, F. (2020, January 20–21). Robotic Grasping using Deep Reinforcement Learning. Proceedings of the 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE), Hong Kong, China.10.1109/CASE48305.2020.9216986 ↗
18. Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., and Funkhouser, T.A. (2018, January 1–5). Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8593986 ↗
19. Kalashnikov, D. (2021, August 17). Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation. Available online: http://proceedings.mlr.press/v87/kalashnikov18a.html.
20. Liu, C., Gao, J., Bi, Y., Shi, X., and Tian, D. (2020). A Multitasking-Oriented Robot Arm Motion Planning Scheme Based on Deep Reinforcement Learning and Twin Synchro-Control. Sensors, 20.10.3390/s20123515 ↗
21. Kerzel, M., Mohammadi, H.B., Zamani, M.A., and Wermter, S. (2018, January 8–13). Accelerating Deep Continuous Reinforcement Learning through Task Simplification. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.10.1109/IJCNN.2018.8489712 ↗
22. Research on robot arm control based on Unity3D machine learning
23. Multi-Target Detection and Grasping Control for Humanoid Robot NAO10.1002/acs.3031 ↗
24. An Automated Fruit Harvesting Robot by Using Deep Learning10.1186/s40648-019-0141-2 ↗
25. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
26. Object Detection through Modified YOLO Neural Network
27. Sang, J., Wu, Z., Guo, P., Hu, H., Xiang, H., Zhang, Q., and Cai, B. (2018). An Improved YOLOv2 for Vehicle Detection. Sensors, 18.10.3390/s18124272 ↗
28. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
29. Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv.
30. Kasper-Eulaers, M., Hahn, N., Berger, S., Sebulonsen, T., Myrland, Ø., and Kummervold, P.E. (2021). Short Communication: Detecting Heavy Goods Vehicles in Rest Areas in Winter Conditions Using YOLOv5. Algorithms, 14.10.3390/a14040114 ↗
31. Kinematics Analysis and Modeling of 6 Degree of Freedom Robotic Arm from DFROBOT on Labview10.19026/rjaset.13.3016 ↗