journal article·2022-12

Vibration control of three coupled flexible beams using reinforcement learning algorithm based on proximal policy optimization

Zhi-cheng Qiu, Jia-hao Du, Xian-min Zhang

Publisher page

Abstract

In order to suppress the vibration of a class of multi-coupling flexible beams, a three-coupling flexible beam experimental device is constructed. The vibration characteristics and active vibration control algorithms of three coupled flexible beams are studied. Due to the complex residual vibration of multi-coupled flexible beam structure, which shows the interaction between beams and close frequency characteristics, the dynamic model of coupled flexible beam structure is established by finite element method (FEM) and its theoretical modal analysis is carried out. Furthermore, the parameters of the experimental system are identified, and the close modal frequencies and damping ratios of the actual system are obtained. The characteristics of close modal vibration are verified. A reinforcement learning (RL) vibration controller based on proximal policy optimization (PPO) algorithm is designed. The algorithm mainly includes actor neural network and critic neural network, which are used to approximate the control strategy and evaluate state respectively. Based on theoretical modeling and experimental identification correction model, the agent learns offline, and then transfers the learned policy parameters to the experimental system for vibration control experiments. The experimental results demonstrate that the designed RL controller can effectively suppress the residual vibration of three coupled flexible beams.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Vibration and Dynamic Analysis · Dynamics and Control of Mechanical Systems · Vibration Control and Rheological Fluids

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Mechanical and Automotive Engineering, South China University of Technology, Guangzhou, PR China

Funding

national natural science foundation of china · natural science foundation of guangdong province · guangzhou municipal science and technology project

References (30)

Show source-provided references
1. 10.1016/j.jsv.2015.10.02910.1016/j.jsv.2015.10.029 ↗
2. Bandyopadhyay B · 2007
3. 10.1561/220000007110.1561/2200000071 ↗
4. 10.1155/2020/290654610.1155/2020/2906546 ↗
5. 10.1109/TSMC.2020.297523210.1109/TSMC.2020.2975232 ↗
6. 10.1016/j.jsv.2020.11527410.1016/j.jsv.2020.115274 ↗
7. 10.1126/science.aaa841510.1126/science.aaa8415 ↗
8. 10.3390/s1611180410.3390/s16111804 ↗
9. 10.1016/j.conengprac.2018.12.01210.1016/j.conengprac.2018.12.012 ↗
10. Lillicrap TP, Hunt JJ, Pritzel A, et al. (2015) Continuous control with deep reinforcement learning. arXiv e-prints. Available at: http://arxiv.org/abs/1509.02971
11. Liu YL, Farimani AB (2021) An energy-saving snake locomotion gait policy obtained using deep reinforcement learning. arXiv e-prints. Available at: https://arxiv.org/abs/2103.04511
12. 10.1016/j.ast.2017.11.03610.1016/j.ast.2017.11.036 ↗
13. 10.1109/TNNLS.2020.297960010.1109/TNNLS.2020.2979600 ↗
14. 10.1016/j.simpat.2016.06.00710.1016/j.simpat.2016.06.007 ↗
15. Mnih V, Kavukcuoglu K, Silver D, et al. (2013) Playing Atari with deep reinforcement learning. arXiv e-prints. Available at: https://arxiv.org/abs/1312.5602
16. Pinto L, Andrychowicz M, Welinder P, et al. (2017) Asymmetric actor critic for image-based robot learning. arXiv e-prints. Available at: http://arxiv.org/abs/1710.0654210.15607/RSS.2018.XIV.008 ↗
17. 10.1115/1.400226610.1115/1.4002266 ↗
18. 10.1016/j.conengprac.2018.08.00110.1016/j.conengprac.2018.08.001 ↗
19. 10.1177/095965181664209910.1177/0959651816642099 ↗
20. Schulman J, Levine S, Moritz P, et al. (2015a) Trust region policy optimization. arXiv e-prints, 2015. Available at: http://arxiv.org/abs/1502.05477
21. Schulman J, Moritz P, Levine S, et al. (2015b) High-dimensional continuous control using generalized advantage estimation. arXiv e-prints. Available at: https://arxiv.org/abs/1506.02438
22. Schulman J, Wolski F, Dhariwal P, et al. (2017) Proximal policy optimization algorithms. arXiv e-prints. Available at: http://arxiv.org/abs/1707.06347
23. Shi F, Homberger T, Lee J, et al. (2020) Circus ANYmal: A quadruped learning dexterous manipulation with its limbs. arXiv e-prints. Available at: https://arxiv.org/abs/2011.0881110.1109/ICRA48506.2021.9561926 ↗
24. Silver D · 2014
25. Sutton RS · 2018
26. Van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double Q-learning. arXiv e-prints. Available at: https://arxiv.org/abs/1509.0646110.1609/aaai.v30i1.10295 ↗
27. 10.1016/j.actaastro.2018.03.02310.1016/j.actaastro.2018.03.023 ↗
28. 10.1016/j.actaastro.2017.07.03610.1016/j.actaastro.2017.07.036 ↗
29. 10.1109/ICoIAS.2019.0003410.1109/ICoIAS.2019.00034 ↗
30. 10.1109/TAES.2017.264925910.1109/TAES.2017.2649259 ↗