Reinforcement Learning-Based PID Gain Optimization for Delta Parallel Robot Trajectory Tracking
Source abstract
Abstract
In this study, a PID gain tuning approach using Deep Deterministic Policy Gradient (DDPG), a reinforcement learning (RL) algorithm, is proposed for trajectory tracking of delta parallel robots. Owing to their 3-degree-of-freedom (3-DOF) parallel kinematic structure, delta robots offer higher stiffness, precision, and speed capabilities than serial manipulators; they are therefore widely used in high-speed pick-and-place applications due to their low moving mass and the stiffness provided by the closed-chain mechanism. In this study, the proposed DDPG-PID approach is comparatively investigated against the conventional Ziegler–Nichols (ZN) and Cohen–Coon (CC) tuning methods; DDPG is designed to optimize the PID gains (Kp, Ki, Kd) within predefined bounds in a continuous action space. In simulations conducted on four different trajectories—circle, lemniscate, diamond, and star—RMSE, IAE, ISE, ITAE, and maximum error metrics are used for evaluation. According to the results, DDPG-PID achieves the lowest error on all trajectories, reducing RMSE by approximately 35–58% compared to ZN-PID and by approximately 79–82% compared to CC-PID; similarly, improvements are observed in IAE/ISE/ITAE and maximum error values. These findings indicate that DDPG-PID provides more stable and accurate tracking, particularly on complex trajectories involving sharp direction changes, and demonstrate that the proposed method offers a superior automatic PID tuning alternative to classical tuning rules for industrial parallel robot control applications.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotic Mechanisms and Dynamics · Dynamics and Control of Mechanical Systems · Iterative Learning Control Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Mechatronics Engineering, Erciyes University, 38039 Kayseri, Türkiye
Mechatronics Engineering, Graduate School of Natural and Applied Sciences, Erciyes University, 38039 Kayseri, Türkiye
References (29)
Show source-provided references
1. Clavel, R. (1988). DELTA, a fast robot with parallel geometry. Proceedings of the 18th International Symposium on Industrial Robots, Springer.
2. Åström, K.J., and Hägglund, T. (1995). PID Controllers: Theory, Design and Tuning, ISA—The Instrumentation, Systems and Automation Society. [2nd ed.].
3. Optimum settings for automatic controllers
4. Theoretical consideration of retarded control
5. Pierrot, F., and Company, O. (1999, January 19–23). H4: A new family of 4-DOF parallel robots. Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Atlanta, GA, USA.10.1109/AIM.1999.803222 ↗
6. Dynamic modeling of parallel robots for computed-torque control implementation10.1177/027836499801701205 ↗
7. Optimization of a three DOF translational platform for well-conditioned workspace10.1109/ROBOT.1997.606784 ↗
8. Biagiotti, L., and Melchiorri, C. (2008). Trajectory Planning for Automatic Machines and Robots, Springer.
9. Merlet, J.P. (2006). Parallel Robots, Springer. [2nd ed.]. Solid Mechanics and Its Applications, 128.
10. Tsai, L.W. (1999). Robot Analysis: The Mechanics of Serial and Parallel Manipulators, John Wiley & Sons.
11. Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, The MIT Press. [2nd ed.].
12. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N.M., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv.
13. Reinforcement learning in robotics: A survey10.1177/0278364913495721 ↗
14. Gu, S., Holly, E., Lillicrap, T., and Levine, S. (June, January 29). Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. Proceedings of the IEEE International Conference on Robotics and Automation, Singapore.
15. Lawrence, N.P., Forbes, M.G., Loewen, P.D., McClement, D.G., Backström, J.U., and Gopaluni, R.B. (2022). Deep reinforcement learning with shallow controllers: An experimental application to PID tuning. Control. Eng. Pract., 121.10.1016/j.conengprac.2021.105046 ↗
16. Adaptive low-level control of autonomous underwater vehicles using deep reinforcement learning10.1016/j.robot.2018.05.016 ↗
17. Delta robot control by learning systems: Harnessing the power of deep reinforcement learning algorithms
18. Yadavari, H., Tavakol Aghaei, V., and İkizoğlu, S. (2023). Deep Reinforcement Learning-Based Control of Stewart Platform With Parametric Simulation in ROS and Gazebo. ASME. J. Mech. Robot., 15.10.1115/1.4056971 ↗
19. Deep Reinforcement Learning Control of Fully-Constrained Cable-Driven Parallel Robots10.1109/TIE.2022.3203763 ↗
20. Evolutionary Optimization of Fuzzy Reinforcement Learning and Its Application to Time-Varying Tracking Control of Industrial Parallel Robotic Manipulators10.1109/TII.2023.3248120 ↗
21. Jalali, H., Samadi, S., Kalhor, A., and Masouleh, M.T. (2022, January 22–24). Model-Free Dynamic Control of a 3-DoF Delta Parallel Robot for Pick-and-Place Application based on Deep Reinforcement Learning. Proceedings of the 10th RSI International Conference on Robotics and Mechatronics (ICRoM), Tehran, Iran.10.1109/ICRoM57054.2022.10025269 ↗
22. Grimshaw, A., and Oyekan, J. (2021). Applying deep reinforcement learning to cable driven parallel robots for balancing unstable loads: A ball case study. Front. Robot. AI, 7.10.3389/frobt.2020.611203 ↗
23. Louette, A., Lambrechts, G., Ernst, D., Pirard, E., and Dislaire, G. (2024). Reinforcement learning to improve delta robot throws for sorting scrap metal. arXiv.
24. Position control of a planar cable-driven parallel robot using reinforcement learning10.1017/S0263574722000273 ↗
25. Compensated Motion and Position Estimation of a Cable-driven Parallel Robot Based on Deep Reinforcement Learning10.1007/s12555-023-0342-6 ↗
26. Azad, F.A., Ansari Rad, S., and Arashpour, M. (2022). Back-stepping control of delta parallel robots with smart dynamic model selection for construction applications. Autom. Constr., 137.10.1016/j.autcon.2022.104211 ↗
27. Åström, K.J., and Hägglund, T. (2006). Advanced PID Control, ISA—The Instrumentation, Systems and Automation Society.
28. Automatic Tuning of Simple Regulators with Specifications on Phase and Amplitude Margins10.1016/0005-1098(84)90014-1 ↗
29. Estimation of time delay time constant parameters in time, frequency, and laplace domains10.1002/cjce.5450560215 ↗