Control Strategy of a Hybrid Renewable Energy System Based on Reinforcement Learning Approach for an Isolated Microgrid
Source abstract
Abstract
Due to the rising cost of fossil fuels and environmental pollution, renewable energy (RE) resources are currently being used as alternatives. To reduce the high dependence of RE resources on the change of weather conditions, a hybrid renewable energy system (HRES) is introduced in this research, especially for an isolated microgrid. In HRES, solar and wind energies are the primary energy resources while the battery and fuel cells (FCs) are considered as the storage systems that supply energy in case of insufficiency. Moreover, a diesel generator is adopted as a back-up system to fulfill the load demand in the event of a power shortage. This study focuses on the development of HRES with the combination of battery and hydrogen FCs. Three major parts were considered including optimal sizing, maximum power point tracking (MPPT) control, and the energy management system (EMS). Recent developments and achievements in the fields of machine learning (ML) and reinforcement learning (RL) have led to new challenges and opportunities for HRES development. Firstly, the optimal sizing of the hybrid renewable hydrogen energy system was defined based on the Hybrid Optimization Model for Multiple Energy Resources (HOMER) software for the case study in an island in the Philippines. According to the assessment of EMS and MPPT control of HRES, it can be concluded that RL is one of the most emerging optimal control solutions. Finally, a hybrid perturbation and observation (P&O) and Q-learning (h-POQL) MPPT was proposed for a photovoltaic (PV) system. It was conducted and validated through the simulation in MATLAB/Simulink. The results show that it showed better performance in comparison to the P&O method.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Hybrid Renewable Energy Systems · Microgrid Control and Optimization · Energy and Environment Impacts
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Aeronautics and Aeronautics, National Cheng Kung University, Tainan 701, Taiwan
Funding
Ministry of Science and Technology, Taiwan
References (48)
Show source-provided references
1. ASEAN Center for Energy Team (2016). ASEAN Renewable Energy Policies, ASEAN Centre for Energy.
2. Lin, C.E., and Phan, B.C. (2016, January 8–10). Optimal Hybrid Energy Solution for Island Micro-Grid. Proceedings of the 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom), Atlanta, GA, USA.10.1109/BDCloud-SocialCom-SustainCom.2016.74 ↗
3. A review on Integrated Renewable Energy System based power generation for stand-alone applications: Configurations, storage options, sizing methodologies and control10.1016/j.rser.2014.05.079 ↗
4. A review of energy management strategies for renewable hybrid energy systems with hydrogen backup10.1016/j.rser.2017.09.014 ↗
5. Control and Management of Hybrid Renewable Energy Systems: Review and Comparison of Methods
6. Multi-objective optimization of a stand-alone hybrid renewable energy system by using evolutionary algorithms: A review10.1016/j.rser.2012.02.071 ↗
7. Artificial intelligence techniques for sizing photovoltaic systems: A review10.1016/j.rser.2008.01.006 ↗
8. A review on planning, configurations, modeling and optimization techniques of hybrid renewable energy systems for off grid applications10.1016/j.rser.2015.12.281 ↗
9. Hybrid renewable microgrid optimization techniques: A review10.1016/j.rser.2017.08.007 ↗
10. General review and classification of different MPPT Techniques10.1016/j.rser.2016.09.132 ↗
11. Energy management strategies in hybrid renewable energy systems: A review10.1016/j.rser.2016.05.040 ↗
12. Sutton, R.S., and Barto, A.G. (2011). Reinforcement Learning: An Introduction, MIT Press.
13. Alpaydin, E. (2014). Introduction to Machine Learning, MIT Press.
14. A survey of the most used MPPT methods: Conventional and advanced algorithms applied for photovoltaic systems10.1016/j.rser.2015.02.009 ↗
15. Maximum Power Point Tracking Approaches for Wind–Solar Hybrid Renewable Energy System—A Review10.1007/978-981-13-0662-4_1 ↗
16. A Reinforcement Learning-Based Maximum Power Point Tracking Method for Photovoltaic Array10.1155/2015/496401 ↗
17. Reinforcement Learning for Online Maximum Power Point Tracking Control10.7763/JOCET.2016.V4.290 ↗
18. A Reinforcement Learning Approach for MPPT Control Method of Photovoltaic Sources10.1016/j.renene.2017.03.008 ↗
19. Resources, configurations, and soft computing techniques for power management and control of PV/wind hybrid system10.1016/j.rser.2016.11.209 ↗
20. Microgrids energy management systems: A critical review on methods, solutions, and prospects10.1016/j.apenergy.2018.04.103 ↗
21. Power Control of PV/Fuel Cell/Supercapacitor Hybrid System for Stand-Alone Applications
22. Energy management based fuzzy logic controller of hybrid system wind/photovoltaic/diesel with storage battery10.1016/j.ijhydene.2017.06.006 ↗
23. Varghese, N., and Reji, P. (2016, January 7–8). Battery charge controller for hybrid stand alone system using adaptive neuro fuzzy inference system. Proceedings of the 2016 International Conference on Energy Efficient Technologies for Sustainability (ICEETS), Nagercoil, India.10.1109/ICEETS.2016.7582920 ↗
24. Hybrid energy storage systems and control strategies for stand-alone renewable energy power systems10.1016/j.rser.2016.07.059 ↗
25. Srinivasan, D., and Jain, L.C. (2010). Multi-agent Reinforcement Learning: An Overview. Innovations in Multi-Agent Systems and Applications-1, Springer.10.1007/978-3-642-14435-6 ↗
26. Nguyen, T., Nguyen, N.D., and Nahavandi, S. (2018). Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications. arXiv.
27. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing Atari with Deep Reinforcement Learning. arXiv.
28. On-line Building Energy Optimization using Deep Reinforcement Learning10.1109/TSG.2018.2834219 ↗
29. Hu, Y., Li, W., Xu, K., Zahid, T., Qin, F., and Li, C. (2018). Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning. Appl. Sci., 8.10.3390/app8020187 ↗
30. Fang, Y., Song, C., Xia, B., and Song, Q. (2015, January 23–25). An energy management strategy for hybrid electric bus based on reinforcement learning. Proceedings of the 27th Chinese Control and Decision Conference (2015 CCDC), Qingdao, China.10.1109/CCDC.2015.7162814 ↗
31. Kim, S., and Lim, H. (2018). Reinforcement Learning Based Energy Management Algorithm for Smart Energy Buildings. Energies, 11.10.3390/en11082010 ↗
32. Leo, R., Milton, R.S., and Sibi, S. (2014, January 26–27). Reinforcement learning for optimal energy management of a solar microgrid. Proceedings of the 2014 IEEE Global Humanitarian Technology Conference-South Asia Satellite (GHTC-SAS), Trivandrum, India.10.1109/GHTC-SAS.2014.6967580 ↗
33. Tan, Y., Liu, W., and Qiu, Q. (2009, January 2–5). Adaptive power management using reinforcement learning. Proceedings of the 2009 IEEE/ACM International Conference on Computer-Aided Design-Digest of Technical Papers, San Jose, CA, USA.10.1145/1687399.1687486 ↗
34. Kofinas, P., Vouros, G., and Dounis, A.I. (2016, January 18–20). Energy Management in Solar Microgrid via Reinforcement Learning. Proceedings of the 9th Hellenic Conference on Artificial Intelligence, Thessaloniki, Greece.10.1145/2903220.2903257 ↗
35. Energy management in solar microgrid via reinforcement learning using fuzzy reward
36. A multi-agent based energy management solution for integrated buildings and microgrid system10.1016/j.apenergy.2017.06.007 ↗
37. Ghorbani, S., Rahmani, R., and Unland, R. (2017). Multi-agent Autonomous Decision Making in Smart Micro-Grids’ Energy Management: A Decentralized Approach. Multiagent System Technologies, Springer International Publishing.10.1007/978-3-319-64798-2_14 ↗
38. Intelligent energy management control for independent microgrid10.1007/s12046-016-0515-6 ↗
39. An Intelligent Multiagent System for Autonomous Microgrid Operation10.3390/en5093347 ↗
40. Multi-Agent System for Distributed Management of Microgrids10.1109/TPWRS.2014.2322622 ↗
41. Event-Triggered-Based Distributed Cooperative Energy Management for Multienergy Systems10.1109/TII.2018.2862436 ↗
42. Distributed Optimal Energy Management for Energy Internet10.1109/TII.2017.2714199 ↗
43. Distributed Optimization of Solar Micro-grid Using Multi Agent Reinforcement Learning10.1016/j.procs.2015.02.016 ↗
44. Fuzzy Q-Learning for multi-agent decentralized energy management in microgrids10.1016/j.apenergy.2018.03.017 ↗
45. Optimal planning of hybrid renewable energy systems using HOMER: A review10.1016/j.rser.2016.05.039 ↗
46. Optimal sizing of hybrid fuel cell-supercapacitor storage system for off-grid renewable applications10.1016/j.energy.2018.10.070 ↗
47. A comprehensive review on solar PV maximum power point tracking techniques10.1016/j.rser.2016.09.076 ↗
48. Reinforcement-Learning-Based Intelligent Maximum Power Point Tracking Control for Wind Energy Conversion Systems10.1109/TIE.2015.2420792 ↗