journal article·2022-09-30Open access

A New Vibration Controller Design Method Using Reinforcement Learning and FIR Filters: A Numerical and Experimental Study

Xingxing Feng, Hong Chen, Gang Wu, Anfu Zhang, Zhigao Zhao

Applied Sciences· Volume 12· Issue 19

Abstract

High-dimensional high-frequency continuous-vibration control problems often have very complex dynamic behaviors. It is difficult for the conventional control methods to obtain appropriate control laws from such complex systems to suppress the vibration. This paper proposes a new vibration controller by using reinforcement learning (RL) and a finite-impulse-response (FIR) filter. First, a simulator with enough physical fidelity was built for the vibration system. Then, the deep deterministic policy gradient (DDPG) algorithm interacted with the simulator to find a near-optimal control policy to meet the specified goals. Finally, the control policy, represented as a neural network, was run directly on a controller in real-world experiments with high-dimensional and high-frequency dynamics. The simulation results show that the maximum peak values of the power-spectrum-density (PSD) curves at specific frequencies can be reduced by over 63%. The experimental results show that the peak values of the PSD curves at specific frequencies were reduced by more than 47% (maximum over 52%). The numerical and experimental results indicate that the proposed controller can significantly attenuate various vibrations within the range from 50 Hz to 60 Hz.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Vibration and Dynamic Analysis · Elevator Systems and Control · Hydraulic and Pneumatic Systems

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Wuhan 2nd Ship Design and Research Institute, Wuhan 430205, China

School of Naval Architecture and Ocean Engineering, Huazhong University of Science and Technology, Wuhan 430074, China

Funding

National Natural Science Foundation of China

References (39)

Show source-provided references
1. Vibration control of semi-active suspension system using PID controller with advanced firefly algorithm and particle swarm optimization10.1007/s12652-020-02158-w ↗
2. Hybrid feedback PID-FxLMS algorithm for active vibration control of cantilever beam with piezoelectric stack actuator10.1016/j.jsv.2021.116243 ↗
3. A novel interval dynamic reliability computation approach for the risk evaluation of vibration active control systems based on PID controllers10.1016/j.apm.2020.11.007 ↗
4. Intelligent control of active shock absorber for high-speed elevator car10.1177/0954406218810045 ↗
5. Laminated piezoelectric beam element for dynamic analysis of piezolaminated smart beams and GA-based LQR active vibration control10.1016/j.compstruct.2020.112480 ↗
6. Takeshita, A., Yamashita, T., Kawaguchi, N., and Kuroda, M. (2021). Fractional-order LQR and state observer for a fractional-order vibratory system. Appl. Sci., 11.10.3390/app11073252 ↗
7. An improved linear quadratic regulator control method through convolutional neural network–based vibration identification10.1177/1077546320933756 ↗
8. Adaptive vibration suppression of time-varying structures with enhanced FxLMS algorithm10.1016/j.ymssp.2018.08.009 ↗
9. Modal filtered-x LMS algorithm for global active noise control in a vibro-acoustic cavity10.1016/j.ymssp.2018.03.031 ↗
10. H∞ Feedback control and Fx-LMS feedforward control for car engine vibration attenuation10.1016/j.apacoust.2004.07.015 ↗
11. Adaptive low-level control of autonomous underwater vehicles using deep reinforcement learning10.1016/j.robot.2018.05.016 ↗
12. Reinforcement learning based compensation methods for robot manipulators10.1016/j.engappai.2018.11.006 ↗
13. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play10.1126/science.aar6404 ↗
14. Ye, D., Liu, Z., Sun, M., Shi, B., Zhao, P., Wu, H., Yu, H., Yang, S., Wu, X., and Guo, Q. (2020). Mastering complex control in MOBA games with deep reinforcement learning. arXiv.10.1609/aaai.v34i04.6144 ↗
15. Magnetic control of tokamak plasmas through deep reinforcement learning10.1038/s41586-021-04301-9 ↗
16. Vibration control of a nonlinear quarter-car active suspension system by reinforcement learning10.1080/00207721.2010.549576 ↗
17. Kim, S.-J., Kim, H.-S., and Kang, D.-J. (2018, January 17–20). Vibration control of a vehicle active suspension system using a DDPG algorithm. Proceedings of the 18th International Conference on Control, Automation and Systems, PyeongChang, Korea.
18. Semi-active suspension control based on deep reinforcement learning10.1109/ACCESS.2020.2964116 ↗
19. Han, S.-Y., and Liang, T. (2022). Reinforcement-learning-based vibration control for a vehicle semi-active suspension system via the PPO approach. Appl. Sci., 12.10.3390/app12063078 ↗
20. Reinforcement learning control of a single-link flexible robotic manipulator10.1049/iet-cta.2016.1540 ↗
21. Reinforcement learning control of a flexible two-link manipulator an experimental investigation10.1109/TSMC.2020.2975232 ↗
22. A vibration control method for hybrid-structured flexible manipulator based on sliding mode control and reinforcement learning10.1109/TNNLS.2020.2979600 ↗
23. Design of model-free reinforcement learning control for tunable vibration absorber system based on magnetorheological elastomer10.1088/1361-665X/abee35 ↗
24. Research on vibration reduction control based on reinforcement learning10.1155/2021/7619214 ↗
25. Reinforcement learning vibration control of a multi-flexible beam coupling system10.1016/j.ast.2022.107801 ↗
26. Trajectory planning and vibration control of translation flexible hinged plate based on optimization and reinforcement learning algorithm10.1016/j.ymssp.2022.109362 ↗
27. Reinforcement learning vibration control for a flexible hinged plate10.1016/j.ast.2021.107056 ↗
28. Watkins, C.J.C.H. (1989). Learning from Delayed Rewards. [Ph.D. Thesis, University of Cambridge].
29. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
30. Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12–17). Deep reinforcement learning with double Q-learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10295 ↗
31. Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2016). Prioritized experience replay. arXiv.
32. Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., and de Freitas, N. (2016). Dueling network architectures for deep reinforcement learning. arXiv.
33. Bellemare, M.G., Dabney, W., and Munos, R. (2017, January 6–11). A distributional perspective on reinforcement learning. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia.
34. Fortunato, M., Azar, M.G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., and Pietquin, O. (2019). Noisy networks for exploration. arXiv.
35. Hessel, M., and Modayil, J. (2018, January 2–7). Rainbow: Combining improvements in deep reinforcement learning. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11796 ↗
36. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2019). Continuous control with deep reinforcement learning. arXiv.
37. Hayes, M.H. (1996). Statistical Digital Signal Processing and Modelling, John Wiley & Sons.
38. On the use of the normalized mean square error in evaluating dispersion model performance10.1016/0960-1686(93)90410-Z ↗
39. Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, The MIT Press.10.1109/TNN.1998.712192 ↗