Reinforcement-Learning-Based Vibration Control for a Vehicle Semi-Active Suspension System via the PPO Approach
Source abstract
Abstract
The vehicle semi-active suspension system plays an important role in improving the driving safety and ride comfort by adjusting the coefficients of the damping and spring. The main contribution of this paper is the proposal of a PPO-based vibration control strategy for a vehicle semi-active suspension system, in which the designed reward function realizes the dynamic adjustment according to the road condition changes. More specifically, for the different suspension performances caused by different road conditions, the three performances of the suspension system, body acceleration, suspension deflection, and dynamic tire load, were taken as the state space of the PPO algorithm, and the reward value was set according to the numerical results of the passive suspension, so that the corresponding damping force was selected as the action space, and the weight matrix of the reward function was dynamically adjusted according to different road conditions, so that the agent could have a better improvement effect at different speeds and road conditions. In this paper, a quarter–car semi-active suspension model was analyzed and simulated, and numerical simulations were performed using stochastic road excitation for different classes of roads, vehicle models, and continuously changing road conditions. The simulation results showed that the body acceleration was reduced by 46.93% under the continuously changing road, which proved that the control strategy could effectively improve the performance of semi-active suspension by combining the dynamic changes of the road with the reward function.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Vibration Control and Rheological Fluids · Vehicle Dynamics and Control Systems · Hydraulic and Pneumatic Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Shandong Provincial Key Laboratory of Network Based Intelligent Computing, University of Jinan, Jinan 250022, China
Funding
Natural Science Foundation of Shandong Province for Key Project · National Natural Science Foundation of China · University Innovation Team Project of Jinan
References (32)
Show source-provided references
1. Semi-active H-infinity control of vehicle suspension with magnetorheological dampers10.1016/j.jsv.2004.05.030 ↗
2. Sky-hook control of nonlinear quarter car model traversing rough road matching performance of LQR control10.1016/j.jsv.2009.01.025 ↗
3. Robust control synthesis for seat suspension systems with actuator saturation and time-varying input delay10.1016/j.jsv.2009.09.017 ↗
4. VFC—Variational feedback controller and its application to semi-active suspensions10.1016/j.ymssp.2016.01.002 ↗
5. Analysis and experiment of a vibration isolator using a novel magnetic spring with negative stiffness10.1016/j.jsv.2014.02.009 ↗
6. A bioinspired dynamics-based adaptive fuzzy SMC method for half-car active suspension systems with input dead zones and saturations10.1109/TCYB.2020.2972322 ↗
7. Bui, Q.-D., and Nguyen, Q.H. (2021, January 15–18). A new approach for dynamic modeling of magnetorheological dampers based on quasi-static model and hysteresis multiplication factor. Proceedings of the IFToMM Asian Conference on Mechanism and Machine Science, Hanoi, Vietnam.10.1007/978-3-030-91892-7_70 ↗
8. Bui, Q., Hoang, L., Mai, D., and Nguyen, Q.H. (2020, January 12–15). Design and testing of a new shear-mode magnetorheological damper with self-power component for front-loaded washing machines. Proceedings of the 2nd Annual International Conference on Material, Machines and Methods for Sustainable Development (MMMS2020), Trang, Vietnam.10.1007/978-3-030-69610-8_114 ↗
9. Robust control for vibration control systems with dead-zone band and time delay under severe disturbance using adaptive fuzzy neural network10.1016/j.jfranklin.2020.09.011 ↗
10. A new optimal sliding mode controller with adjustable gains based on Bolza–Meyer criterion for vibration control10.1016/j.jsv.2020.115542 ↗
11. Mnih, V., Kavukcuoglu, K., and Silver, D. (2013). Playing Atari with deep reinforcement learning. arXiv.
12. Lillicrap, T.P., Hunt, J.J., and Pritzel, A. (2015). Continuous control with deep reinforcement learning. arXiv.
13. Schulman, J., Levine, S., and Moritz, P. (2015, January 6–11). Trust region policy optimization. Proceedings of the 2015 32nd International Conference on Machine Learning, Lille, France.
14. Mastering the game of Go with deep neural networks and tree search10.1038/nature16961 ↗
15. Mastering complex control in MOBA games with deep reinforcement learning
16. Schulman, J., and Wolski, F.P. (2017). Proximal policy optimization algorithms. arXiv.
17. Image captioning via proximal policy optimization10.1016/j.imavis.2021.104126 ↗
18. Sadhukhan, P., and Selmic, R.R. (2021, January 17–20). Multi-agent formation control with obstacle avoidance using proximal policy optimization. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC), Melbourne, Australia.10.1109/SMC52423.2021.9658635 ↗
19. Tognetti, S., Savaresi, S.M., and Spelta, C. (2009, January 8–10). Batch reinforcement learning for semi-active suspension control. Proceedings of the 2009 IEEE Control Applications, (CCA) and Intelligent Control, St. Petersburg, Russia.10.1109/CCA.2009.5281070 ↗
20. Vibration control of a nonlinear quarter-car active suspension system by reinforcement learning10.1080/00207721.2010.549576 ↗
21. Li, Z., Chu, T., and Kalabic, U. (2019, January 19–21). Dynamics-enabled safe deep reinforcement learning: Case study on active suspension control. Proceedings of the IEEE Conference on Control Technology and Applications (CCTA), Hong Kong, China.10.1109/CCTA.2019.8920696 ↗
22. Zhao, F., Jiang, P., You, K., Song, S., Zhang, W., and Tong, L. (2019, January 16–19). Setpoint tracking for the suspension system of medium-speed maglev trains via reinforcement learning. Proceedings of the 2019 IEEE 15th International Conference on Control and Automation (ICCA), Edinburgh, UK.10.1109/ICCA.2019.8900006 ↗
23. Semi-active suspension control based on deep reinforcement learning10.1109/ACCESS.2020.2964116 ↗
24. Technical Committee ISO/TC, Mechanical Vibration, Shock (1995). Subcommittee SC2 Measurement, Mechanical vibration-road surface profiles-reporting of measured data. Int. Organ. Stand., 8608, Available online: https://kns.cnki.net/kcms/detail/detail.aspx?FileName=SCSF00013909&DbName=SCSF.
25. Numerical simulation of stochastic road process using white noise filtration10.1016/j.ymssp.2005.01.009 ↗
26. Simulation of passive suspension system for improving ride comfort of vehicle
27. A survey on multi-agent deep reinforcement learning: From the perspective of challenges and applications10.1007/s10462-020-09938-y ↗
28. Obando-Ceron, J.S., and Castro, P.S. (2021, January 18–24). Revisiting rainbow: Promoting more insightful and inclusive deep reinforcement learning research. Proceedings of the 2021 International Conference on Machine Learning (ICML), Online.
29. Reinforcement learning based optimal control of batch processes using Monte-Carlo deep deterministic policy gradient with phase segmentation10.1016/j.compchemeng.2020.107133 ↗
30. Zimmer, M., and Weng, P. (2019, January 10–16). Exploiting the sign of the advantage function to learn deterministic policies in continuous domains. Proceedings of the 2019 International Joint Conferences on Artificial Intelligence (IJCAI), Macao, China.10.24963/ijcai.2019/625 ↗
31. Zhang, H., Bai, S., and Lan, X. (2021, January 19–27). Hindsight trust region policy optimization. Proceedings of the 2019 International Joint Conference on Artificial Intelligence (IJCAI), Montreal, QC, Canada.10.24963/ijcai.2021/459 ↗
32. An improved enhanced fireworks algorithm based on adaptive explosion amplitude and levy flight