Fuzzy-Partitioned Multi-Agent TD3 for Photovoltaic Maximum Power Point Tracking Under Partial Shading
Source abstract
Abstract
Maximum power point tracking (MPPT) under partial shading is a nonconvex, rapidly varying control problem that challenges multi-agent policies deployed on photovoltaic modules. We present Fuzzy–MAT3D, a fuzzy-augmented multi-agent TD3 (Twin-Delayed Deep Deterministic Policy Gradient) controller trained under centralized training/decentralized execution (CTDE). On the theory side, we prove that differentiable fuzzy partitions of unity endow the actor–critic maps with global Lipschitz regularity, reduce temporal-difference target variance, enlarge the input-to-state stability (ISS) margin, and yield a global L∞γ-contraction of fixed-policy evaluation (hence, non-expansive with κ=γ<1). We further state a two-time-scale convergence theorem for CTDE-TD3 with fuzzy features; a PL/last-layer-linear corollary implies point convergence and uniqueness of critics. We bound the projected Bellman residual with the correct contraction factor (for L∞ and L2(ρ) under measure invariance) and quantified the negative bias induced by min{Q1,Q2}; an N-agent extension is provided. Empirically, a balanced common-random-numbers design across seven scenarios and 20 seeds, analyzed by ANOVA and CRN-paired tests, shows that Fuzzy–MAT3D attains the highest mean MPPT efficiency (92.0% ± 4.0%), outperforming MAT3D and Multi-Agent Deep Deterministic Policy Gradient controller (MADDPG). Overall, fuzzy regularization yields higher efficiency, suppresses steady-state oscillations, and stabilizes learning dynamics, supporting the use of structured, physics-compatible features in multi-agent MPPT controllers. At the level of PV plants, such gains under partial shading translate into higher effective capacity factors and smoother renewable generation without additional hardware.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Photovoltaic System Optimization Techniques · Smart Grid Energy Management · Solar Radiation and Photovoltaics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Industrial Engineering and Manufacturing, Universidad Autónoma de Ciudad Juárez, Ciudad Juárez 32310, Chihuahua, Mexico
References (52)
Show source-provided references
1. Comparison of photovoltaic array maximum power point tracking techniques10.1109/TEC.2006.874230 ↗
2. A critical evaluation on maximum power point tracking methods for partial shading in PV systems10.1016/j.rser.2015.03.080 ↗
3. A Comparative Study on Maximum Power Point Tracking Techniques for Photovoltaic Power Systems10.1109/TSTE.2012.2202294 ↗
4. Comparative Study of Maximum Power Point Tracking Algorithms10.1002/pip.459 ↗
5. Comprehensive Approach to Modeling and Simulation of Photovoltaic Arrays10.1109/TPEL.2009.2013862 ↗
6. A review of global maximum power point tracking techniques for photovoltaic systems under partial shading conditions—A survey10.1016/j.rser.2018.04.094 ↗
7. An Improved Particle Swarm Optimization (PSO)–Based MPPT for PV With Reduced Steady-State Oscillation10.1109/TPEL.2012.2185713 ↗
8. A deep recurrent reinforcement learning approach for enhanced MPPT in PV systems10.1016/j.asoc.2024.111728 ↗
9. Deep reinforcement learning using deep-Q-network for Global Maximum Power Point tracking: Design and experiments in real photovoltaic systems10.1016/j.heliyon.2024.e37974 ↗
10. Physics-Informed Multi-Agent Deep Reinforcement Learning Enabled Distributed Voltage Control for Active Distribution Network Using PV Inverters10.1016/j.ijepes.2023.109641 ↗
11. Alsulami, A.A.G., Alhussainy, A.A., Allehyani, A., Alturki, Y.A., Alghamdi, S.M., Alruwaili, M., and Alharthi, Y.Z. (2024). A comparison of several maximum power point tracking algorithms for a photovoltaic power system. Front. Energy Res., 12.10.3389/fenrg.2024.1439264 ↗
12. Endiz, M.S., Gökkuş, G., Coşgun, A.E., and Demir, H. (2025). A Review of Traditional and Advanced MPPT Approaches for PV Systems Under Uniformly Insolation and Partially Shaded Conditions. Appl. Sci., 15.10.3390/app15031031 ↗
13. Siddique, M.A.B., Zhao, D., Ouahada, K., Rehman, A.U., and Hamam, H. (2025). Performance validation of global MPPT for efficient power extraction through PV system under complex partial shading effects. Sci. Rep., 15.10.1038/s41598-025-01816-3 ↗
14. Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 21–26). Deterministic Policy Gradient Algorithms. Proceedings of the 31st International Conference on Machine Learning (ICML), Beijing, China. Available online: https://proceedings.mlr.press/v32/silver14.html.
15. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016). Continuous control with deep reinforcement learning. arXiv.
16. Fujimoto, S., van Hoof, H., and Meger, D. (2018, January 10–15). Addressing function approximation error in actor-critic methods. Proceedings of the 35th International Conference on Machine Learning (ICML), Stockholm, Sweden. Available online: https://proceedings.mlr.press/v80/fujimoto18a.html.
17. Deep Reinforcement Learning with Double Q-Learning
18. Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mordatch, I. (2017). Multi-Agent Actor-Critic for Mixed Cooperative–Competitive Environments. Proceedings of the Advances in Neural Information Processing Systems, Curran Associates, Inc.
19. Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., and Whiteson, S. (2018, January 2–7). Counterfactual Multi-Agent Policy Gradients. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-18), New Orleans, LA, USA.10.1609/aaai.v32i1.11794 ↗
20. Rashid, T., Samvelyan, M., de Witt, C.S., Farquhar, G., Foerster, J., and Whiteson, S. (2018, January 10–15). QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning. Proceedings of the 35th International Conference on Machine Learning (ICML), Stockholm, Sweden. Available online: http://jmlr.org/papers/v21/20-081.html.
21. Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10–15). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. Proceedings of the 35th International Conference on Machine Learning (ICML), Stockholm, Sweden. Available online: https://proceedings.mlr.press/v80/haarnoja18b.html.
22. Schulman, J., Levine, S., Moritz, P., Jordan, M.I., and Abbeel, P. (2015, January 6–11). Trust Region Policy Optimization. Proceedings of the 32nd International Conference on Machine Learning (ICML), Lille, France. Available online: https://proceedings.mlr.press/v37/schulman15.html.
23. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv.
24. Konda, V.R., and Tsitsiklis, J.N. (2000). Actor–Critic Algorithms. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), MIT Press.
25. On Actor–Critic Algorithms10.1137/S0363012901385691 ↗
26. The O.D.E. method for convergence of stochastic approximation and reinforcement learning10.1137/S0363012997331639 ↗
27. Borkar, V.S. (2025). Stochastic Approximation: A Dynamical Systems Viewpoint, Springer.
28. Natural actor–critic algorithms10.1016/j.automatica.2009.07.008 ↗
29. Fuzzy sets10.1016/S0019-9958(65)90241-X ↗
30. Bezdek, J.C. (1981). Pattern Recognition with Fuzzy Objective Function Algorithms, Springer.10.1007/978-1-4757-0450-1 ↗
31. ANFIS: Adaptive-network-based fuzzy inference system10.1109/21.256541 ↗
32. The partition of unity finite element method: Basic theory and applications10.1016/S0045-7825(96)01087-0 ↗
33. The Partition of Unity Method10.1002/(SICI)1097-0207(19970228)40:4<727::AID-NME86>3.0.CO;2-N ↗
34. Shepard, D. (1968, January 27–29). A two-dimensional interpolation function for irregularly-spaced data. Proceedings of the 23rd ACM National Conference, New York, NY, USA.10.1145/800186.810616 ↗
35. Input to state stability: Basic concepts and results10.1007/978-3-540-77653-6_3 ↗
36. Khalil, H.K. (2002). Nonlinear Systems, Prentice Hall. [3rd ed.].
37. Bauschke, H.H., and Combettes, P.L. (2017). Correction to: Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer International Publishing.10.1007/978-3-319-48311-5 ↗
38. Scherrer, B. (2010, January 21–24). Should one compute the temporal difference fix point or minimize the Bellman residual? The unified oblique projection view. Proceedings of the 27th International Conference on Machine Learning (ICML), Haifa, Israel. Available online: https://icml.cc/Conferences/2010/papers/654.pdf.
39. Bertsekas, D.P., and Tsitsiklis, J.N. (1996). Neuro-Dynamic Programming, Athena Scientific.
40. An analysis of temporal-difference learning with function approximation10.1109/9.580874 ↗
41. Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., and Meger, D. (2018, January 2–7). Deep Reinforcement Learning that Matters. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), New Orleans, LA, USA. Available online: https://dl.acm.org/doi/abs/10.5555/3504035.3504427.10.1609/aaai.v32i1.11694 ↗
42. Agarwal, R., Schwarzer, M., Castro, P.S., Courville, A., and Bellemare, M.G. (2025, September 04). Deep Reinforcement Learning at the Edge of the Statistical Precipice. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS). Available online: https://arxiv.org/pdf/2108.13264.
43. A multiple comparison procedure for comparing several treatments with a control10.1080/01621459.1955.10501294 ↗
44. Controlling the false discovery rate: A practical and powerful approach to multiple testing10.1111/j.2517-6161.1995.tb02031.x ↗
45. Law, A.M., and Kelton, W.D. (2014). Simulation Modeling and Analysis, McGraw-Hill. [5th ed.].
46. Safety-Enhanced Self-Learning for Optimal Power Converter Control10.1109/TIE.2024.3363759 ↗
47. Ren, H., Wang, Y., Yu, H., Zhang, B., and Chen, Z. (2024, January 17–20). Deep Reinforcement Learning-based Power Flow Control for Triple Active Bridge Converter. Proceedings of the 2024 IEEE 10th International Power Electronics and Motion Control Conference (IPEMC2024-ECCE Asia), Chengdu, China.10.1109/IPEMC-ECCEAsia60879.2024.10568017 ↗
48. Reinforcement Learning-Based Predictive Control for Power Electronic Converters10.1109/TIE.2024.3472299 ↗
49. A Critical Review of Safe Reinforcement Learning Strategies in Power and Energy Systems10.1016/j.engappai.2025.110091 ↗
50. Abouzeid, A.F., Eleraky, H., Kalas, A., Rizk, R., Elsakka, M.M., and Refaat, A. (2024). Experimental Validation of a Low-Cost Maximum Power Point Tracking Technique Based on Artificial Neural Network for Photovoltaic Systems. Sci. Rep., 14.10.1038/s41598-024-67306-0 ↗
51. Ahmadi, M., Abrari, M., Ghanaatshoar, M., and Khalafi, A. (2024). A Novel Algorithm for Maximum Power Point Tracking Using Computer Vision (CVMPPT). PLoS ONE, 19.10.1371/journal.pone.0301363 ↗
52. Teneta, J., Kreft, W., and Janowski, M. (2024). Partial Shading of Photovoltaic Modules with Thin Linear Objects: Modelling in MATLAB Environment and Measurement Experiments. Energies, 17.10.3390/en17143546 ↗