Coverage Path Planning with Semantic Segmentation for UAV in PV Plants
Source abstract
Abstract
Solar energy is one of the most strategic energy sources for the world’s economic development. This has caused the number of solar photovoltaic plants to increase around the world; consequently, they are installed in places where their access and manual inspection are arduous and risky tasks. Recently, the inspection of photovoltaic plants has been conducted with the use of unmanned aerial vehicles (UAV). Although the inspection with UAVs can be completed with a drone operator, where the UAV flight path is purely manual or utilizes a previously generated flight path through a ground control station (GCS). However, the path generated in the GCS has many restrictions that the operator must supply. Due to these restrictions, we present a novel way to develop a flight path automatically with coverage path planning (CPP) methods. Using a DL server to segment the region of interest (RoI) within each of the predefined PV plant images, three CPP methods were also considered and their performances were assessed with metrics. The UAV energy consumption performance in each of the CPP methods was assessed using two different UAVs and standard metrics. Six experiments were performed by varying the CPP width, and the consumption metrics were recorded in each experiment. According to the results, the most effective and efficient methods are the exact cellular decomposition boustrophedon and grid-based wavefront coverage, depending on the CPP width and the area of the PV plant. Finally, a relationship was established between the size of the photovoltaic plant area and the best UAV to perform the inspection with the appropriate CPP width. This could be an important result for low-cost inspection with UAVs, without high-resolution cameras on the UAV board, and in small plants.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotics and Sensor-Based Localization · Robotic Path Planning Algorithms · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Research Group in Efficient Energy Management (GIMEL), Electrical Engineering Department, Universidad de Antioquia, Calle 67 No. 53-108, Medellín 050010, Colombia
Funding
Colombia Scientific Program
References (100+)
Show source-provided references
1. REN21® (2021, August 14). GLOBAL STATUS REPORTS. Available online: https://www.ren21.net/reports/global-status-report/.
2. Alkhraijah, M., Alowaifeer, M., Alsaleh, M., Alfaris, A., and Molzahn, D.K. (2021). The Effects of Social Distancing on Electricity Demand Considering Temperature Dependency. Energies, 14.10.3390/en14020473 ↗
3. Mey, A. (2021, October 03). Most U.S. Utility-Scale Solar Photovoltaic Power Plants Are 5 Megawatts or Smaller-Today in Energy-U.S. Energy Information Administration (EIA), Available online: https://www.eia.gov/todayinenergy/detail.php?id=38272#.
4. Maghrabie, H.M., Abdelkareem, M.A., Al-Alami, A.H., Ramadan, M., Mushtaha, E., Wilberforce, T., and Olabi, A.G. (2021). State-of-the-Art Technologies for Building-Integrated Photovoltaic Systems. Buildings, 11.10.3390/buildings11090383 ↗
5. A novel method for parameter identification and performance estimation of PV module under varying operating conditions10.1016/j.enconman.2021.114689 ↗
6. IRENA (2021, October 24). Future of Solar Photovoltaic: Deployment, Investment, Technology, Grid Integration, and Socio-Economic Aspects. Available online: https:https://www.irena.org.
7. Survey on PV modules common faults after an O and M flight extensive campaign over different plants in Italy10.1109/JPHOTOV.2017.2674977 ↗
8. PV Panel and PV Inverter Damages Caused by Combination of Edge Delamination, Water Penetration, and High String Voltage in Moderate Climate10.1109/JPHOTOV.2021.3050984 ↗
9. Performance degradation of photovoltaic power system: Review on mitigation methods10.1016/j.rser.2016.09.072 ↗
10. Kaplani, E. (2016). PV cell and module degradation, detection and diagnostics. Renewable Energy in the Service of Mankind Vol II, Selected Topics from the World Renewable Energy Congress WREC, London, UK, 3–8 August 2014, Springer.10.1007/978-3-319-18215-5_35 ↗
11. Review of O and M Practices in PV Plants: Failures, Solutions, Remote Control, and Monitoring Tools10.1109/JPHOTOV.2020.2994531 ↗
12. Donovan, C.W. (2020). Renewable Energy Finance: Funding the Future of Energy, World Scientific Publishing Co. Pte. Ltd.10.1142/q0254 ↗
13. Solutions for adapting photovoltaics to large power irrigation systems for agriculture
14. Grimaccia, F., Leva, S., Niccolai, A., and Cantoro, G. (2018, January 12–15). Assessment of PV plant monitoring system by means of unmanned aerial vehicles. Proceedings of the 2018 IEEE International Conference on Environment and Electrical Engineering and 2018 IEEE Industrial and Commercial Power Systems Europe (EEEIC/I&CPS Europe), Palermo, Italy.10.1109/EEEIC.2018.8494532 ↗
15. Guerrero-Liquet, G.C., Oviedo-Casado, S., Sánchez-Lozano, J., García-Cascales, M.S., Prior, J., and Urbina, A. (2018). Determination of the Optimal Size of Photovoltaic Systems by Using Multi-Criteria Decision-Making Methods. Sustainability, 10.10.20944/preprints201811.0267.v1 ↗
16. Planning for PV plant performance monitoring by means of unmanned aerial systems (UAS)10.1007/s40095-014-0149-6 ↗
17. Shen, K., Qiu, Q., Wu, Q., Lin, Z., and Wu, Y. (2019, January 1–3). Research on the Development Status of Photovoltaic Panel Cleaning Equipment Based on Patent Analysis. Proceedings of the 2019 3rd International Conference on Robotics and Automation Sciences (ICRAS), Wuhan, China.10.1109/ICRAS.2019.8809015 ↗
18. Azaiz, R. (2017). Flying Robot for Processing and Cleaning Smooth, Curved and Modular Surfaces. (App. 15/118,849), U.S. Patent.
19. Libra, M., Daneček, M., Lešetickỳ, J., Poulek, V., Sedláček, J., and Beránek, V. (2019). Monitoring of Defects of a Photovoltaic Power Plant Using a Drone. Energies, 12.10.3390/en12050795 ↗
20. An unmanned inspection system for multiple defects detection in photovoltaic plants10.1109/JPHOTOV.2019.2955183 ↗
21. Alsafasfeh, M., Abdel-Qader, I., Bazuin, B., Alsafasfeh, Q., and Su, W. (2018). Unsupervised fault detection and analysis for large photovoltaic systems using drones and machine vision. Energies, 11.10.3390/en11092252 ↗
22. Gallardo-Saavedra, S., Franco-Mejia, E., Hernández-Callejo, L., Duque-Pérez, Ó., Loaiza-Correa, H., and Alfaro-Mejia, E. (November, January 29). Aerial thermographic inspection of photovoltaic plants: Analysis and selection of the equipment. Proceedings of the 2017 Proceedings ISES Solar World Congress, IEA SHC, Abu Dhabi, UAE.10.18086/swc.2017.20.03 ↗
23. Leva, S., Aghaei, M., and Grimaccia, F. (2015, January 10–13). PV power plant inspection by UAS: Correlation between altitude and detection of defects on PV modules. Proceedings of the 2015 IEEE 15th International Conference on Environment and Electrical Engineering (EEEIC), Rome, Italy.10.1109/EEEIC.2015.7165466 ↗
24. Aghaei, M., Dolara, A., Leva, S., and Grimaccia, F. (2016, January 17–21). Image resolution and defects detection in PV inspection by unmanned technologies. Proceedings of the 2016 IEEE Power and Energy Society General Meeting (PESGM), Boston, MA, USA.10.1109/PESGM.2016.7741605 ↗
25. Light unmanned aerial vehicles (UAVs) for cooperative inspection of PV plants10.1109/JPHOTOV.2014.2323714 ↗
26. Fault inspection by aerial infrared thermography in a pv plant after a meteorological tsunami
27. Fault diagnosis of photovoltaic modules through image processing and Canny edge detection on field thermographic measurements10.1080/14786451.2013.826223 ↗
28. Niccolai, A., Grimaccia, F., and Leva, S. (2019). Advanced Asset Management Tools in Photovoltaic Plant Monitoring: UAV-Based Digital Mapping. Energies, 12.10.3390/en12244736 ↗
29. The orthophoto and orthophotomap: Characteristics, development and application10.1559/152304076784080249 ↗
30. Yao, Y.Y., and Hu, Y.T. (2017, January 16–18). Recognition and location of solar panels based on machine vision. Proceedings of the 2017 2nd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS), Wuhan, China.10.1109/ACIRS.2017.7986055 ↗
31. Leva, S., and Aghaei, M. (2018). Power Engineering: Advances and Challenges Part B: Electrical Power. Chapter 3: Failures and Defects in PV Systems Review and Methods of Analysis, Taylor & Francis Group, CRC Press.
32. Automatic photovoltaic panel area extraction from uav thermal infrared images
33. deep learning Based Module Defect Analysis for Large-Scale Photovoltaic Farms10.1109/TEC.2018.2873358 ↗
34. Ding, Y., Cao, R., Liang, S., Qi, F., Yang, Q., and Yan, W. (2020, January 22–24). Density-Based Optimal UAV Path Planning for Photovoltaic Farm Inspection in Complex Topography. Proceedings of the 2020 Chinese Control And Decision Conference (CCDC), Hefei, China.10.1109/CCDC49329.2020.9164257 ↗
35. Luo, X., Li, X., Yang, Q., Wu, F., Zhang, D., Yan, W., and Xi, Z. (2017, January 20–22). Optimal path planning for UAV based inspection system of large-scale photovoltaic farm. Proceedings of the 2017 Chinese Automation Congress (CAC), Jinan, China.10.1109/CAC.2017.8243572 ↗
36. Salahat, E., Asselineau, C.A., Coventry, J., and Mahony, R. (2019, January 14–17). Waypoint Planning for Autonomous Aerial Inspection of Large-Scale Solar Farms. Proceedings of the IECON 2019-45th Annual Conference of the IEEE Industrial Electronics Society, Lisbon, Portugal.10.1109/IECON.2019.8927123 ↗
37. Sizkouhi, A.M.M., Esmailifar, S.M., Aghaei, M., De Oliveira, A.K.V., and Rüther, R. (2019, January 16–21). Autonomous path planning by unmanned aerial vehicle (UAV) for precise monitoring of large-scale PV plants. Proceedings of the 2019 IEEE 46th Photovoltaic Specialists Conference (PVSC), Chicago, IL, USA.
38. Choset, H.M., Lynch, K.M., Hutchinson, S., Kantor, G., Burgard, W., Kavraki, L., Thrun, S., and Arkin, R.C. (2005). Principles of Robot Motion: Theory, Algorithms, and Implementation, MIT Press.
39. Cabreira, T.M., Brisolara, L.B., and Ferreira Jr, P.R. (2019). Survey on coverage path planning with unmanned aerial vehicles. Drones, 3.10.3390/drones3010004 ↗
40. Sebbane, Y.B. (2018). Intelligent Autonomy of UAVs: Advanced Missions and Future Use, CRC Press.
41. Coverage for robotics–a survey of recent results10.1023/A:1016639210559 ↗
42. Motion planner for a Tetris-inspired reconfigurable floor cleaning robot10.1177/1729881420914441 ↗
43. Coombes, M., Fletcher, T., Chen, W.H., and Liu, C. (2018). Optimal Polygon Decomposition for UAV Survey coverage path planning in Wind. Sensors, 18.10.3390/s18072132 ↗
44. Islam, N., Rashid, M.M., Pasandideh, F., Ray, B., Moore, S., and Kadel, R. (2021). A Review of Applications and Communication Technologies for Internet of Things (IoT) and unmanned aerial vehicle (UAV) Based Sustainable Smart Farming. Sustainability, 13.10.3390/su13041821 ↗
45. Pham, H.X., La, H.M., Feil-Seifer, D., and Deans, M. (2017, January 24–28). A distributed control framework for a team of unmanned aerial vehicles for dynamic wildfire tracking. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.10.1109/IROS.2017.8206579 ↗
46. Coverage trajectory planning for a bush trimming robot arm10.1002/rob.21917 ↗
47. Chang, W., Yang, G., Yu, J., Liang, Z., Cheng, L., and Zhou, C. (2017, January 24–28). Development of a power line inspection robot with hybrid operation modes. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.10.1109/IROS.2017.8202263 ↗
48. Cooperative coverage path planning for visual inspection10.1016/j.conengprac.2018.03.002 ↗
49. A survey on coverage path planning for robotics10.1016/j.robot.2013.09.004 ↗
50. Bormann, R., Jordan, F., Hampp, J., and Hägele, M. (2018, January 21–25). Indoor coverage path planning: Survey, implementation, analysis. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.10.1109/ICRA.2018.8460566 ↗
51. Nam, L., Huang, L., Li, X., and Xu, J. (2016, January 22–24). An approach for coverage path planning for UAVs. Proceedings of the 2016 IEEE 14th International Workshop on Advanced Motion Control, AMC 2016, Auckland, New Zealand.10.1109/AMC.2016.7496385 ↗
52. Quality-aware UAV coverage and path planning in geometrically complex environments10.1016/j.adhoc.2018.02.008 ↗
53. Time-optimal trajectory generation for aerial coverage of urban building10.1016/j.ast.2018.10.011 ↗
54. Majeed, A., and Lee, S. (2019). A new coverage flight path planning algorithm based on footprint sweep fitting for unmanned aerial vehicle navigation in urban environments. Appl. Sci., 9.10.3390/app9071470 ↗
55. Elfes, A., Campos, M., Bergerman, M., Bueno, S., and Podnar, G. (2000, January 25–30). A robotic unmanned aerial vehicle for environmental research and monitoring. Proceedings of the First Scientific Conference on the Large Scale Biosphere-Atmosphere Experiment in Amazonia (LBA), LBA Central Office, CPTEC/INPE, Belém, Pará, Brazil.
56. On realistic target coverage by autonomous drones10.1145/3325512 ↗
57. Lingelbach, F. (May, January 26). Path planning using probabilistic cell decomposition. Proceedings of the IEEE International Conference on Robotics and Automation, 2004. Proceedings, ICRA’04, New Orleans, LA, USA.10.1109/ROBOT.2004.1307193 ↗
58. coverage path planning Techniques for Inspection of Disjoint Regions With Precedence Provision10.1109/ACCESS.2020.3044987 ↗
59. Khiati, W., Moumen, Y., Habchi, A.E., Zerrouk, I., Berrich, J., and Bouchentouf, T. (2020, January 21–23). Grid Based approach (GBA): A new approach based on the grid-clustering algorithm to solve a CPP type problem for air surveillance using UAVs. Proceedings of the 2020 Fourth International Conference On Intelligent Computing in Data Sciences (ICDS), Fez, Morocco.10.1109/ICDS50568.2020.9268683 ↗
60. A comparison of path planning strategies for autonomous exploration and mapping of unknown environments10.1007/s10514-012-9298-8 ↗
61. An efficient Harris hawks-inspired image segmentation method10.1016/j.eswa.2020.113428 ↗
62. A survey on deep learning techniques for image and video semantic segmentation10.1016/j.asoc.2018.05.018 ↗
63. Long, J., Shelhamer, E., and Darrell, T. (2015, January 7–12). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA.10.1109/CVPR.2015.7298965 ↗
64. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.10.1007/978-3-319-24574-4_28 ↗
65. Pérez-González, A., Jaramillo-Duque, A., and Cano-Quintero, J.B. (2021). Automatic Boundary Extraction for Photovoltaic Plants Using the Deep Learning UNet Model. Appl. Sci., 11.10.3390/app11146524 ↗
66. Automatic boundary extraction of large-scale photovoltaic plants using a fully convolutional network on aerial imagery10.1109/JPHOTOV.2020.2992339 ↗
67. coverage path planning for UAVs photogrammetry with energy and resolution constraints10.1007/s10846-016-0348-x ↗
68. Zefri, Y., ElKettani, A., Sebari, I., and Ait Lamallam, S. (2018). Thermal infrared and visual inspection of photovoltaic installations by UAV photogrammetry application case: Morocco. Drones, 2.10.3390/drones2040041 ↗
69. Coopmans, C., Podhradskỳ, M., and Hoffer, N.V. (2015, January 23–25). Software-and hardware-in-the-loop verification of flight dynamics model and flight control simulation of a fixed-wing unmanned aerial vehicle. Proceedings of the 2015 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS), Cancun, Mexico.10.1109/RED-UAS.2015.7440998 ↗
70. Roggi, G., Niccolai, A., Grimaccia, F., and Lovera, M. (2020). A Computer Vision Line-Tracking Algorithm for Automatic UAV Photovoltaic Plants Monitoring Applications. Energies, 13.10.3390/en13040838 ↗
71. Coates, E.M., and Fossen, T.I. (2021). Geometric Reduced-Attitude Control of Fixed-Wing UAVs. Appl. Sci., 11.10.3390/app11073147 ↗
72. Tullu, A., Endale, B., Wondosen, A., and Hwang, H.Y. (2021). Machine Learning Approach to Real-Time 3D Path Planning for Autonomous Navigation of unmanned aerial vehicle. Appl. Sci., 11.10.3390/app11104706 ↗
73. Pitonakova, L., Giuliani, M., Pipe, A., and Winfield, A. (2018). Feature and performance comparison of the V-REP, Gazebo and ARGoS robot simulators. Annual Conference Towards Autonomous Robotic Systems, Springer.10.1007/978-3-319-96728-8_30 ↗
74. Akcakoca, M., Atici, B.M., Gever, B., Oguz, S., Demirezen, U., Demir, M., Saldiran, E., Yuksek, B., Koyuncu, E., and Yeniceri, R. (2019). A simulation-based development and verification architecture for micro uav teams and swarms. AIAA Scitech 2019 Forum, AIAA.10.2514/6.2019-1979 ↗
75. (2021, October 11). Unoeste Terá Maior Usina Solar de Geração Distribuída de SP-Unoeste. Available online: http://www.unoeste.br/noticias/2019/3/unoeste-tera-maior-usina-solar-de-geracao-distribuida-de-sp.
76. (2021, October 11). ABA Newsletter. Available online: http://www.csus.edu/aba2/newsletters/fall2012/abagreennews.html.
77. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016). TensorFlow: A System for Large-Scale Machine Learning. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), USENIX Association.
78. (2021, October 31). Flask. Available online: https://flask.palletsprojects.com/en/2.0.x/.
79. (2021, October 24). Mavlink/Mavlink. Available online: https://github.com/mavlink/mavlink.
80. Li, J., Zhou, Y., and Lamont, L. (2013, January 9–13). Communication architectures and protocols for networking unmanned aerial vehicles. Proceedings of the 2013 IEEE Globecom Workshops (GC Wkshps), Atlanta, GA, USA.
81. Machine Learning: Algorithms, Real-World Applications and Research Directions10.1007/s42979-021-00592-x ↗
82. The openCV library
83. (2021, October 18). OpenCV: OpenCV-Python Tutorials. Available online: https://docs.opencv.org/master/d6/d00/tutorial_py_root.html.
84. Choset, H., and Pignon, P. (1998). coverage path planning: The boustrophedon cellular decomposition. Field and Service Robotics, Springer.10.1007/978-1-4471-1273-0_32 ↗
85. Spanning-tree based coverage of continuous areas by a mobile robot10.1023/A:1016610507833 ↗
86. Zelinsky, A., Jarvis, R.A., Byrne, J., and Yuta, S. (1993, January 8–9). Planning paths of complete coverage of an unstructured environment by a mobile robot. Proceedings of the 1993 IEEE/Tsukuba International Workshop on Advanced Robotics, Tsukuba, Japan.
87. Simulation and assessment educational framework for mobile robot algorithms
88. Jaeyoung, L. (2021, December 07). PX4 Gazebo Plugin Suite for MAVLink SITL and HITL. Available online: https://github.com/PX4/PX4-SITL_gazebo/blob/ffb87ef4a312564cf91791bd5a9d683aacd085a6/models/iris/iris.sdf.jinja.
89. coverage path planning with unmanned aerial vehicles for 3D terrain reconstruction10.1016/j.eswa.2016.02.007 ↗
90. Araújo, J., Sujit, P., and Sousa, J. (2013, January 16–19). Multiple UAV area decomposition and coverage. Proceedings of the 2013 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), Singapore.10.1109/CISDA.2013.6595424 ↗
91. (2021, September 23). PX4 Drone Autopilot. Available online: https://github.com/PX4/PX4-Autopilot.
92. Koenig, N., and Howard, A. (October, January 28). Design and use paradigms for Gazebo, an open-source multi-robot simulator. Proceedings of the 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566), Sendai, Japan.
93. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., and Ng, A.Y. (2009, January 12–17). ROS: An open-source Robot Operating System. Proceedings of the ICRA Workshop on Open Source Software, Kobe, Japan.10.1109/MRA.2010.936956 ↗
94. Koubaa, A. (2020). CrazyS: A Software-in-the-Loop Simulation Platform for the Crazyflie 2.0 Nano-Quadcopter. Robot Operating System (ROS): The Complete Reference (Volume 4), Springer International Publishing.
95. Salamh, F.E., Karabiyik, U., and Rogers, M.K. (2019). RPAS Forensic Validation Analysis Towards a Technical Investigation Process: A Case Study of Yuneec Typhoon H. Sensors, 19.10.3390/s19153246 ↗
96. (2021, October 31). QGroundControl Ground Control Station. Available online: https://github.com/mavlink/qgroundcontrol.
97. (2021, October 28). Yuneec Typhoon H-Yuneec Futurhobby. Available online: https://yuneec-futurhobby.com/yuneec-typhoon-h-pro-realsense.
98. (2021, October 28). 3DR Iris-The Ready to fly UAV Quadcopter. Available online: http://www.arducopter.co.uk/iris-quadcopter-uav.html.
99. Tan, S.H., and Md Ali, J. (2012, January 18–20). Quartic and quintic polynomial interpolation. Proceedings of the 20th National Symposium on Mathematical Sciences: Research in Mathematical Sciences: A Catalyst for Creativity and Innovation, Putrajaya, Malaysia.10.1063/1.4801189 ↗
100. Peréz-González, A. (2021, October 29). Andresperez86/CPP_GUI. Available online: https://github.com/andresperez86/CPP_GUI.