Online Reinforcement Learning-Based Control of an Active Suspension System Using the Actor Critic Approach
Source abstract
Abstract
In this paper, a controller learns to adaptively control an active suspension system using reinforcement learning without prior knowledge of the environment. The Temporal Difference (TD) advantage actor critic algorithm is used with the appropriate reward function. The actor produces the actions, and the critic criticizes the actions taken based on the new state of the system. During the training process, a simple and uniform road profile is used while maintaining constant system parameters. The controller is tested using two road profiles: the first one is similar to the one used during the training, while the other one is bumpy with an extended range. The performance of the controller is compared with the Linear Quadratic Regulator (LQR) and optimum Proportional-Integral-Derivative (PID), and the adaptiveness is tested by estimating some of the system’s parameters using the Recursive Least Squares method (RLS). The results show that the controller outperforms the LQR in terms of the lower overshoot and the PID in terms of reducing the acceleration.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Vibration Control and Rheological Fluids · Hydraulic and Pneumatic Systems · Vehicle Dynamics and Control Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Mechatronics Engineering, Jordan University of Science and Technology, Irbid 22110-3030, Jordan
Aerospace Engineering, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-1308, USA
Mechanical Engineering, Al Hussein Technical University, Amman 11831-139, Jordan
References (21)
Show source-provided references
1. Robust adaptive control of an active vehicle suspension system10.1243/0954407991526621 ↗
2. A sliding mode controller for vehicle active suspension systems with non-linearities10.1243/0954407981525812 ↗
3. Ekoru, J.E., Dahunsi, O.A., and Pedro, J.O. (2011, January 13–15). Pid control of a nonlinear half-car active suspension system via force feedback. Proceedings of the IEEE Africon’11, Livingstone, Zambia.10.1109/AFRCON.2011.6071979 ↗
4. Talib, M.H.A., and Darns, I.Z.M. (2013, January 7–9). Self-tuning pid controller for active suspension system with hydraulic actuator. Proceedings of the 2013 IEEE Symposium on Computers & Informatics (ISCI), Langkawi, Malaysia.
5. Fuzzy control of a quarter-car suspension system
6. Mittal, R. (2015, January 24–26). Robust pi and lqr controller design for active suspension system with parametric uncertainty. Proceedings of the 2015 International Conference on Signal Processing, Computing and Control (ISPCC), Waknaghat, India.10.1109/ISPCC.2015.7375007 ↗
7. H∞ control of active suspension system for a quarter car model
8. A neural network based sliding mode controller for active vehicle suspension10.1243/09544070JAUTO242 ↗
9. Design a pid controller for suspension system by back propagation neural network
10. Zhao, F., Dong, M., Qin, Y., Gu, L., and Guan, J. (2015). Adaptive neural networks control for camera stabilization with active suspension system. Adv. Mech. Eng., 7.10.1177/1687814015599926 ↗
11. Road excitation classification for semi-active suspension system based on system response10.1177/1077546317693432 ↗
12. Deep learning framework for controlling an active suspension system10.1177/1077546319853070 ↗
13. Stochastic optimal control of active vehicle suspensions using learning automata
14. Application of learning automata to controller design in slow-active automobile suspensions10.1080/00423119508969109 ↗
15. Moderated reinforcement learning of active and semi-active vehicle suspension control laws
16. Vibration control of a nonlinear quarter-car active suspension system by reinforcement learning10.1080/00207721.2010.549576 ↗
17. Chen, H.C., Lin, Y.C., and Chang, Y.H. (2018, January 4–7). An actor-critic reinforcement learning control approach for discrete-time linear system with uncertainty. Proceedings of the 2018 International Automatic Control Conference (CACS), Taoyuan, Taiwan.10.1109/CACS.2018.8606740 ↗
18. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv.
19. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
20. Åström, K.J., and Wittenmark, B. (2013). Adaptive Control, Courier Corporation.
21. Apkarian, J., and Abdossalami, A. (2013). Active Suspension Experiment for Matlabr/Simulinkr Users—Laboratory Guide, Quanser Inc.