Deep Deterministic Policy Gradient to Regulate Feedback Control Systems Using Reinforcement Learning
Source abstract
Abstract
Controlling feedback control systems in continuous action spaces has always been a challenging problem. Nevertheless, reinforcement learning is mainly an area of artificial intelligence (AI) because it has been used in process control for more than a decade. However, the existing algorithms are unable to provide satisfactory results. Therefore, this research uses a reinforcement learning (RL) algorithm to manage the control system. We propose an adaptive speed control of the motor system based on depth deterministic strategy gradient (DDPG). The actor-critic scenario using DDPG is implemented to build the RL agent. In addition, a framework has been created for traditional feedback control systems to make RL implementation easier for control systems. The RL algorithms are robust and proficient in using trial and error to search for the best strategy. Our proposed algorithm is a deep deterministic policy gradient, in which a large amount of training data trains the agent. Once the system is trained, the agent can automatically adjust the control parameters. The algorithm has been developed using Python 3.6 and the simulation results are evaluated in the MATLAB/Simulink environment. The performance of the proposed RL algorithm is compared with a proportional integral derivative (PID) controller and a linear quadratic regulator (LQR) controller. The simulation results of the proposed scheme are promising for the feedback control problems.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Reinforcement Learning in Robotics · Adaptive Dynamic Programming Control · Smart Grid Security and Resilience
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (27)
Show source-provided references
1. Deep learning10.1038/nature14539 ↗
2. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou et al., “Playing Atari with deep reinforcement learning. New York, ArXiv preprint, arXiv:1312.5602, 2013.
3. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
4. Mastering the game of Go with deep neural networks and tree search10.1038/nature16961 ↗
5. K. Arulkumaran, M. Deisenroth, M. Brundage and A. Bharath, “A brief survey of deep reinforcement learning. New York, ArXiv preprint arXiv:1708.05866, 2017.10.1109/MSP.2017.2743240 ↗
6. T. Lillicrap, J. Hunt, A. Pritzel, N. Heess, T. Erez et al., “Proximal policy optimization algorithms learning. New York, ArXiv preprint arXiv:1509.02971, 2015.
7. Optimal tracking control for completely unknown nonlinear discrete-time Markov jump systems using data-based reinforcement learning method10.1016/j.neucom.2016.02.029 ↗
8. An intelligent non-integer PID controller-based deep reinforcement learning: Implementation and experimental results10.1109/TIE.2020.2979561 ↗
9. Deep reinforcement learning based optimal trajectory tracking control of autonomous underwater vehicle
10. Impact of inertial response for the variable speed wind turbine
11. Control strategy of speed servo systems based on deep reinforcement learning10.3390/a11050065 ↗
12. Design and analysis of robust fuzzy logic maximum power point tracking based isolated photovoltaic energy system10.1002/eng2.12234 ↗
13. W. Shi, S. Song and C. Wu, “Soft policy gradient method for maximum entropy deep reinforcement learning. New York, ArXiv preprint arXiv:1909.03198, 2019.10.24963/ijcai.2019/475 ↗
14. A novel approach to feedback control with deep reinforcement learning10.1016/j.ifacol.2018.09.241 ↗
15. Intelligent greenhouse monitoring and control scheme: An arrangement of Sensors, Raspberry Pi based Embedded System and IoT platform10.17485/IJST/v13i27.311 ↗
16. Deep ensemble reinforcement learning with multiple deep deterministic policy gradient algorithm
17. Sutton · Finite Markov Decision Processes in Reinforcement Learning · 2017 · 1 · 37
18. Actor-critic algorithms10.1137/S0363012901385691 ↗
19. T. Lillicrap, J. Hunt, A. Pritzel, N. Heess, T. Erez et al., “Continuous control with deep reinforcement learning. New York, ArXiv preprint arXiv:1509.02971, 2015.
20. Trust region policy optimization
21. J. Schulman, F. Wolski, P. Dhariwal, A. Radford and O. Klimov, “Proximal policy optimization algorithms. New York, ArXiv preprint arXiv:1707.06347, 2017.
22. Z. Wang, V. Bapst, N. Heess, V. Mnih, R. Munos et al., “Sample efficient actor-critic with experience replay. New York, ArXiv preprint arXiv:1611.01224, 2016.
23. Hindsight experience replay
24. Dueling network architectures for deep reinforcement learning
25. Continuous control with deep reinforcement learning
26. Deep reinforcement learning approaches for process control
27. On replacing PID controller with deep learning controller for DC motor system10.12720/joace.3.6.452-456 ↗