journal article·2025-05-02Open access

Soft Fuzzy Reinforcement Neural Network Proportional–Derivative Controller

Qiang Han, Farid Boussaid, Mohammed Bennamoun

Applied Sciences· Volume 15· Issue 9

Abstract

Controlling systems with highly nonlinear or uncertain dynamics present significant challenges, particularly when using conventional Proportional–Integral–Derivative (PID) controllers, as they can be difficult to tune. While PID controllers can be adapted for such systems using advanced tuning methods, they often struggle with lag and instability due to their integral action. In contrast, fuzzy Proportional–Derivative (PD) controllers offer a more responsive alternative by eliminating reliance on error accumulation and enabling rule-based adaptability. However, their industrial adoption remains limited due to challenges in manual rule design. To overcome this limitation, Fuzzy Neural Networks (FNNs) integrate neural networks with fuzzy logic, enabling self-learning and reducing reliance on manually crafted rules. However, most fuzzy neural network PD (FNNPD) controllers rely on mean square error (MSE)-based training, which can be inefficient and unstable in complex, dynamic systems. To address these challenges, this paper presents a Soft Fuzzy Reinforcement Neural Network PD (SFPD) controller, integrating the Soft Actor–Critic (SAC) framework into FNNPD control to improve training speed and stability. While the actor–critic framework is widely used in reinforcement learning, its application to FNNPD controllers has been unexplored. The proposed controller leverages reinforcement learning to autonomously adjust parameters, eliminating the need for manual tuning. Additionally, entropy-regularized stochastic exploration enhances learning efficiency. It can operate with or without expert knowledge, leveraging neural network-driven adaptation. While expert input is not required, its inclusion accelerates convergence and improves initial performance. Experimental results show that the proposed SFPD controller achieves fast learning, superior control performance, and strong robustness to noise, making it effective for complex control tasks.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Sensor and Control Systems

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Electrical, Electronic & Computer Engineering, The University of Western Australia, Perth 6009, Australia

Department of Computer Science and Software Engineering, The University of Western Australia, Perth 6009, Australia

Funding

Australian Research Council (ARC DP210101682) and the UWA Research Collaboration Award

References (45)

Show source-provided references
1. Patents, software, and hardware for PID control: An overview and analysis of the current art10.1109/MCS.2006.1580153 ↗
2. An optimal fuzzy PID controller10.1109/41.937407 ↗
3. Fuzzy-PID-based trajectory tracking for 3WIS robot
4. Design of brushless DC motor simulation action system based on Fuzzy PID
5. PD with sliding mode control for trajectory tracking of robotic system10.1016/j.rcim.2013.09.009 ↗
6. Enhanced P-type control: Indirect adaptive learning from set-point updates10.1109/TAC.2022.3154347 ↗
7. Fuzzy PID controller: Design, performance evaluation, and stability analysis10.1016/S0020-0255(99)00127-9 ↗
8. Laib, A., and Gharib, M. (2024). Design of an Intelligent Cascade Control Scheme Using a Hybrid Adaptive Neuro-Fuzzy PID Controller for the Suppression of Drill String Torsional Vibration. Appl. Sci., 14.10.3390/app14125225 ↗
9. A comparative study on PI-PD controller design using stability region centroid methods for unstable, integrating and resonant systems with time delay10.1177/00202940241253114 ↗
10. Bhattacharjee, D., Kim, W., Chattopadhyay, A., Waser, R., and Rana, V. (2018). Multi-valued and fuzzy logic realization using TaOx memristive devices. Sci. Rep., 8.10.1038/s41598-017-18329-3 ↗
11. A robust self-tuning scheme for PI-and PD-type fuzzy controllers10.1109/91.746295 ↗
12. Kumar, P., Nema, S., and Padhy, P. (2014, January 8–10). Design of Fuzzy Logic based PD Controller using cuckoo optimization for inverted pendulum. Proceedings of the 2014 IEEE International Conference on Advanced Communications, Control and Computing Technologies, Ramanathapuram, India.10.1109/ICACCCT.2014.7019346 ↗
13. Zhuang, Q., Xiao, M., Tao, B., and Cheng, S. (2021, January 22–24). Bifurcations control of IS-LM macroeconomic system via PD controller. Proceedings of the 2021 China Automation Congress (CAC), Beijing, China.10.1109/CAC53003.2021.9728545 ↗
14. Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., and Abbeel, P. (2018). Soft actor-critic algorithms and applications. arXiv.
15. He, J., Su, S., Wang, H., Chen, F., and Yin, B. (2023). Online PID Tuning Strategy for Hydraulic Servo Control Systems via SAC-Based Deep Reinforcement Learning. Machines, 11.10.3390/machines11060593 ↗
16. A self-adaptive SAC-PID control approach based on reinforcement learning for mobile robots10.1002/rnc.5662 ↗
17. Song, L., Xu, C., Hao, L., Yao, J., and Guo, R. (2022). Research on PID parameter tuning and optimization based on SAC-auto for USV path following. J. Mar. Sci. Eng., 10.10.3390/jmse10121847 ↗
18. Advances and challenges in learning from experience replay10.1007/s10462-024-11062-0 ↗
19. Di-Castro, S., Mannor, S., and Di Castro, D. (2022, January 17–23). Analysis of stochastic processes through replay buffers. Proceedings of the International Conference on Machine Learning, Baltimore, MD, USA.
20. Tuning fuzzy PD and PI controllers using reinforcement learning10.1016/j.isatra.2010.05.005 ↗
21. Puriel-Gil, G., Yu, W., and Sossa, H. (2018, January 5–7). Reinforcement learning compensation based PD control for inverted pendulum. Proceedings of the 2018 15th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE), Mexico City, Mexico.10.1109/ICEEE.2018.8533946 ↗
22. Reinforcement learning compensation based PD control for a double inverted pendulum10.1109/TLA.2019.8863179 ↗
23. Optimization of Fuzzy-PD Control for a 3-DOF Robotics Manipulator Using a Back-Propagation Neural Network10.18280/mmep.110122 ↗
24. Neural Network based Fuzzy plus PD Controller for Speed Control in Electric Vehicles
25. McCutcheon, L., and Fallah, S. (2023, January 1–5). Adaptive PD Control using Deep Reinforcement Learning for Local-Remote Teleoperation with Stochastic Time Delays. Proceedings of the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Detroit, MI, USA.10.1109/IROS55552.2023.10341953 ↗
26. A fuzzy neural network and its application to pattern recognition10.1109/91.298447 ↗
27. A novel fuzzy neural network based distance relaying scheme10.1109/61.871350 ↗
28. Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press. A Bradford Book.
29. Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10–15). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
30. Backpropagation and stochastic gradient descent method10.1016/0925-2312(93)90006-O ↗
31. Kingma, D.P., and Welling, M. (2013). Auto-Encoding Variational Bayes. arXiv.
32. Körpeoğlu, S.G., Filiz, A., and Yıldız, S.G. (2025). AI-driven predictions of mathematical problem-solving beliefs: Fuzzy logic, adaptive neuro-fuzzy inference systems, and artificial neural networks. Appl. Sci., 15.10.3390/app15020494 ↗
33. Villa-Ávila, E., Arévalo, P., Ochoa-Correa, D., Espinoza, J.L., Albornoz-Vintimilla, E., and Jurado, F. (2025). Improving V2G Systems Performance with Low-Pass Filter and Fuzzy Logic for PV Power Smoothing in Weak Low-Voltage Networks. Appl. Sci., 15.10.3390/app15041952 ↗
34. Lu, W., Liang, J., and Su, H. (2025). Research on Energy-Saving Optimization Method and Intelligent Control of Refrigeration Station Equipment Based on Fuzzy Neural Network. Appl. Sci., 15.10.3390/app15031077 ↗
35. Speedup of implementing fuzzy neural networks with high-dimensional inputs through parallel processing on graphic processing units10.1109/TFUZZ.2011.2140326 ↗
36. Enhancing active vibration control performances in a smart rotary sandwich thick nanostructure conveying viscous fluid flow by a PD controller10.1080/17455030.2021.1948627 ↗
37. Yang, R., Gao, Y., Wang, H., and Ni, X. (2023). Fuzzy neural network PID control used in Individual blade control. Aerospace, 10.10.3390/aerospace10070623 ↗
38. Prado, R., Melo, J., Oliveira, J., and Neto, A.D. (2012, January 10–15). FPGA based implementation of a Fuzzy Neural Network modular architecture for embedded systems. Proceedings of the The 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, QLD, Australia.10.1109/IJCNN.2012.6252447 ↗
39. Comparison of tuning methods of PID controller
40. Optimized fuzzy PD cascade controller: A comparative analysis and design10.1016/j.simpat.2010.06.004 ↗
41. Entropy-maximizing TD3-based reinforcement learning for adaptive PID control of dynamical systems10.1016/j.compchemeng.2023.108393 ↗
42. Chowdhury, M.A., and Lu, Q. (June, January 31). A novel entropy-maximizing TD3-based reinforcement learning for automatic PID tuning. Proceedings of the 2023 American Control Conference (ACC), San Diego, CA, USA.10.23919/ACC55779.2023.10156246 ↗
43. Reinforcement learning approach to autonomous PID tuning10.1016/j.compchemeng.2022.107760 ↗
44. Adaptive neuro-fuzzy PID controller based on twin delayed deep deterministic policy gradient algorithm10.1016/j.neucom.2020.03.063 ↗
45. Reinforcement learning based adaptive PID controller design for control of linear/nonlinear unstable processes10.1016/j.asoc.2022.109450 ↗