journal article·2024-01-19Open access

Cascaded Fuzzy Reward Mechanisms in Deep Reinforcement Learning for Comprehensive Path Planning in Textile Robotic Systems

Di Zhao, Zhenyu Ding, Wenjie Li, Sen Zhao, Yuhong Du

Applied Sciences· Volume 14· Issue 2

Abstract

With the rapid advancement of industrial automation and artificial intelligence technologies, particularly in the textile industry, robotic technology is increasingly challenged with intelligent path planning and executing high-precision tasks. This study focuses on the automatic path planning and yarn-spool-assembly tasks of textile robotic arms, proposing an end-to-end planning and control model that integrates deep reinforcement learning. The innovation of this paper lies in the introduction of a cascaded fuzzy reward system, which is integrated into the end-to-end model to enhance learning efficiency and reduce ineffective exploration, thereby accelerating the convergence of the model. A series of experiments conducted in a simulated environment demonstrate the model’s exceptional performance in yarn-spool-assembly tasks. Compared to traditional reinforcement learning methods, our model shows potential advantages in improving task success rates and reducing collision rates. The cascaded fuzzy reward system, a core component of our end-to-end deep reinforcement learning model, offers a novel and more robust solution for the automated path planning of robotic arms. In summary, the method proposed in this study provides a new perspective and potential applications for industrial automation, especially in the operation of robotic arms in complex and uncertain environments.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Robot Manipulation and Learning · Robotic Path Planning Algorithms · Prosthetics and Rehabilitation Robotics

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Mechanical Engineering, Tiangong University, Tianjin 300387, China

Engineering Teaching Practice training Center of Tiangong University, Tianjin 300387, China

State Key Laboratory of Turbulence and Complex Systems, College of Engineering of Peking University, Beijing 100871, China

Intelligent Bionic Design Laboratory, College of Engineering of Peking University, Beijing 100871, China

School of Electronics & Information Engineering, Tiangong University, Tianjin 300387, China

Innovation College, Tiangong University, Tianjin 300387, China

Funding

Tianjin Science and Technology Bureau · Ministry of Education of the People’s Republic of China

References (40)

Show source-provided references
1. Comparing Task Simplifications to Learn Closed Loop Object Picking Using Deep Reinforcement Learning10.1109/LRA.2019.2896467 ↗
2. Distributed Learning of Decentralized Control Policies for Articulated Mobile Robots10.1109/TRO.2019.2922493 ↗
3. Reward Functions for Learning to Control in Air Traffic Flow Management10.1016/j.trc.2013.06.010 ↗
4. Reinforcement Learning Based on Movement Primitives for Contact Tasks—ScienceDirect10.1016/j.rcim.2019.101863 ↗
5. Optimizing Deep Learning Parameters Using Genetic Algorithm for Object Recognition and Robot Grasping
6. Generating Quality Grasp Rectangle Using Pix2Pix GAN for Intelligent Robot Grasping10.1007/s00138-022-01362-2 ↗
7. Frasson, C. (October, January 30). On the Development of a Personalized Augmented Reality Spatial Ability Training Mobile Application//Novelties in Intelligent Digital Systems. Proceedings of the 1st International Conference (NIDS 2021), Athens, Greece. Available online: https://ebooks.iospress.nl/doi/10.3233/FAIA210078.
8. Research on Hand-Eye Calibration Technology of Visual Service Robot Grasping Based On
9. Sangiovanni, B., Rendiniello, A., and Incremona, G.P. (2018). European Control Conference (ECC), IEEE.
10. Mahmood, A.R., Korenkevych, D., Komer, B.J., and Bergstra, J. (2018). Setting up a Reinforcement Learning Task with a Real-World Robot. arXiv.
11. Wen, S., Chen, J., Wang, S., Zhang, H., and Hu, X. (2018, January 12–15). Path Planning of Humanoid Arm Based on Deep Deterministic Policy Gradient. Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia.10.1109/ROBIO.2018.8665248 ↗
12. A Survey on Transfer Learning10.1109/TKDE.2009.191 ↗
13. Feedback Deep Deterministic Policy Gradient with Fuzzy Reward for Robotic Multiple Peg-in-Hole Assembly Tasks10.1109/TII.2018.2868859 ↗
14. Research Progress in Robot Motion Control Based on Deep Reinforcement Learning
15. Kalashnikov, D., Irpan, A., and Pastor, P. (2018, January 29–31). Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation//Conference on Robot Learning. Proceedings of the Conference on Robot Learning, Zürich, Switzerland.
16. Yahya, A., Li, A., Kalakrishnan, M., Chebotar, Y., and Levine, S. (2017). 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE.
17. Learning Positioning Policies for Mobile Manipulation Operations with Deep Reinforcement Learning10.1007/s13042-023-01815-8 ↗
18. Bridging the Reality Gap between Virtual and Physical Environments through Reinforcement Learning10.1109/ACCESS.2023.3249572 ↗
19. Finn, C., Levine, S., and Abbeel, P. (2016). Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization. arXiv.
20. Ho, J., and Ermon, S. (2016). Generative Adversarial Imitation Learning. arXiv.
21. Towards End-to-End Formation Control for Robotic Fish via Deep Reinforcement Learning with Non-Expert Imitation10.1016/j.oceaneng.2023.113811 ↗
22. Peng, X.B., Abbeel, P., Levine, S., and van de Panne, M. (2018). DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills. arXiv.10.1145/3197517.3201311 ↗
23. Escontrela, A., Peng, X.B., Yu, W., Zhang, T., Iscen, A., Goldberg, K., and Abbeel, P. (2022, January 23–27). Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions. Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.10.1109/IROS47612.2022.9981973 ↗
24. Energy Management in Solar Microgrid via Reinforcement Learning Using Fuzzy Reward10.1080/17512549.2017.1314832 ↗
25. Deep Reinforcement Learning Based Trajectory Planning under Uncertain Constraints10.3389/fnbot.2022.883562 ↗
26. A Review on the Applications of Type-2 Fuzzy Logic in Classification and Pattern Recognition10.1016/j.eswa.2013.03.020 ↗
27. An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller10.1006/ijhc.1973.0303 ↗
28. Structure Identification of Fuzzy Model10.1016/0165-0114(88)90113-3 ↗
29. Fuzzy Mathematical Models in Engineering and Management Science10.1080/00401706.1990.10484661 ↗
30. The Nearest Trapezoidal Fuzzy Number to a Fuzzy Quantity
31. Microsoft Kinect V2 Vision System in a Manufacturing Application. Robot10.1016/j.rcim.2017.04.001 ↗
32. Hierarchical Policies for Cluttered-Scene Grasping with Latent Plans10.1109/LRA.2022.3143198 ↗
33. Guo, M., Wang, Y., Liang, B., Chen, Z., Lin, J., and Huang, K. (2022). Lecture Notes in Electrical Engineering, Springer.
34. Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., and Funkhouser, T. (2018, January 1–5). Learning Synergies between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.10.1109/IROS.2018.8593986 ↗
35. Deng, Y., Guo, X., Wei, Y., Lu, K., Fang, B., Guo, D., Liu, H., and Sun, F. (2019, January 4–8). Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment. Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.10.1109/IROS40897.2019.8967899 ↗
36. An Overview of Evolutionary Algorithms in Multiobjective Optimization10.1162/evco.1995.3.1.1 ↗
37. End-to-End Navigation Strategy with Deep Reinforcement Learning for Mobile Robots10.1109/TII.2019.2936167 ↗
38. High Utility Periodic Frequent Pattern Mining in Multiple Sequences
39. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous Control with Deep Reinforcement Learning. arXiv.
40. Wang, S., Cao, Y., Zheng, X., and Zhang, T. (2021, January 26–28). An End-to-End Trajectory Planning Strategy for Free-Floating Space Robots. Proceedings of the 2021 40th Chinese Control Conference (CCC), Shanghai, China.10.23919/CCC52363.2021.9550509 ↗