Design of an Autonomous Cooperative Drone Swarm for Inspections of Safety Critical Infrastructure
Source abstract
Abstract
Inspection of critical infrastructure with drones is experiencing an increasing uptake in the industry driven by a demand for reduced cost, time, and risk for inspectors. Early deployments of drone inspection services involve manual drone operations with a pilot and do not obtain the technological benefits concerning autonomy, coordination, and cooperation. In this paper, we study the design needed to handle the complexity of an Unmanned Aerial System (UAS) to support autonomous inspection of safety-critical infrastructure. We apply a constructive research approach to link innovation needs with concepts, designs, and validations that include simulation and demonstration of key design parts. Our design approach addresses the complexity of the UAS and provides a selection of technology components for drone and ground control hardware and software including algorithms for autonomous operation and interaction with cloud services. The paper presents a drone perception system with accelerated onboard computing, communication technologies of the UAS, as well as algorithms for swarm membership, formation flying, object detection, and fault detection with artificial intelligence. We find that the design of a cooperative drone swarm and its integration into a custom-built UAS for infrastructure inspection is highly feasible given the current state of the art in electronic components, software, and communication technology.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: UAV Applications and Optimization · Robotics and Sensor-Based Localization · Distributed Control Multi-Agent Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical and Computer Engineering, Aarhus University, DK-8200 Aarhus N, Denmark
Department of Mathematics and Computer Science, University of Southern Denmark, DK-5230 Odense M, Denmark
The Maersk Mc-Kinney Moller Institute, SDU UAS Center, University of Southern Denmark, DK-5230 Odense M, Denmark
Department of Technology Entrepreneurship and Innovation, University of Southern Denmark, DK-6400 Sønderborg, Denmark
Funding
European Union’s Horizon
References (57)
Show source-provided references
1. Review of unmanned aircraft system (UAS)
2. Use of UAS for damage inspection and assessment of bridge infrastructures10.1016/j.ijdrr.2022.102824 ↗
3. Cacace, J., Orozco-Soto, S.M., Suarez, A., Caballero, A., Orsag, M., Bogdan, S., Vasiljevic, G., Ebeid, E., Rodriguez, J.A.A., and Ollero, A. (2021). Safe Local Aerial Manipulation for the Installation of Devices on Power Lines: AERIAL-CORE First Year Results and Designs. Appl. Sci., 11.10.3390/app11136220 ↗
4. Review of Unmanned Aerial System (UAS) applications in the built environment: Towards automated building inspection procedures using drones10.1016/j.autcon.2018.05.002 ↗
5. Santos, T., Moreira, M., Almeida, J., Dias, A., Martins, A., Dinis, J., Formiga, J., and Silva, E. (2017, January 26–28). PLineD: Vision-based power lines detection for Unmanned Aerial Vehicles. Proceedings of the 2017 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Coimbra, Portugal.10.1109/ICARSC.2017.7964084 ↗
6. Utilization of Unmanned Aerial Vehicle, Artificial Intelligence, and Remote Measurement Technology for Bridge Inspections10.20965/jrm.2020.p1244 ↗
7. A Method for Optimized Deployment of a Network of Surveillance Aerial Drones10.1109/JSYST.2019.2910080 ↗
8. Elmeseiry, N., Alshaer, N., and Ismail, T. (2021). A Detailed Survey and Future Directions of Unmanned Aerial Vehicles (UAVs) with Potential Applications. Aerospace, 8.10.3390/aerospace8120363 ↗
9. Kimchi, G., Buchmueller, D., Green, S.A., Beckman, B.C., Isaacs, S., Navot, A., Hensel, F., Bar-Zeev, A., and Rault, S.S.J.M. (2015). Unmanned Aerial Vehicle Delivery System. (WO2015061008A1).
10. Traveling salesman problem with a drone station10.1109/TSMC.2018.2867496 ↗
11. Development and prospect of unmanned aerial vehicle technologies for agricultural production management
12. Distributed Connectivity Maintenance in Swarm of Drones During Post-Disaster Transportation Applications10.1109/TITS.2021.3066843 ↗
13. The drone ambulance [A-UAS]: Golden bullet or just a blank?10.1016/j.resuscitation.2017.04.037 ↗
14. Cajo, R., Thi, T.M., Copot, C., Plaza, D., Keyser, R.D., and Ionescu, C. (2019, January 6–9). Multiple UAVs Formation for Emergency Equipment and Medicines Delivery Based on Optimal Fractional Order Controllers. Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy.10.1109/SMC.2019.8914431 ↗
15. Sanders, R. (2003). An israeli military innovation: UAVs. JFQ, 114–118.
16. (2022, July 18). Drones4Safety Project Home Page. Available online: http://drones4safety.eu.
17. Iversen, N., Schofield, O.B., Cousin, L., Ayoub, N., Bogel, G.V., and Ebeid, E. (2021). Design, Integration and Implementation of an Intelligent and Self-recharging Drone System for Autonomous Power Line Inspection, Institute of Electrical and Electronics Engineers Inc.10.1109/IROS51168.2021.9635924 ↗
18. Achievements and Challenges in Machine Vision-Based Inspection of Large Concrete Structures10.1260/1369-4332.17.3.303 ↗
19. Bruns, I. (2023, January 03). Drones for Power Line Inspections. Available online: https://www.utilityproducts.com/line-construction-maintenance/article/16003823/drones-for-power-line-inspections.
20. Nakatsu, R., and Hoshino, J. (2003). Entertainment Computing: Technologies and Application, Springer.10.1007/978-0-387-35660-0 ↗
21. Cantieri, A., Ferraz, M., Szekir, G., Antônio Teixeira, M., Lima, J., Schneider Oliveira, A., and Aurélio Wehrmeister, M. (2020). Cooperative UAV–UGV Autonomous Power Pylon Inspection: An Investigation of Cooperative Outdoor Vehicle Positioning Architecture. Sensors, 20.10.3390/s20216384 ↗
22. UAVs that fly forever: Uninterrupted structural inspection through automatic UAV replacement10.1016/j.adhoc.2017.11.012 ↗
23. Castrillo, V.U., Manco, A., Pascarella, D., and Gigante, G. (2022). A Review of Counter-UAS Technologies for Cooperative Defensive Teams of Drones. Drones, 6.10.3390/drones6030065 ↗
24. Myjak, M.V.K., and Ranganathan, P. (2022, January 19–21). Unmanned Aerial System (UAS) Swarm Design, Flight Patterns, Communication Type, Applications, and Recommendations. Proceedings of the 2022 IEEE International Conference on Electro Information Technology (eIT), Mankato, MN, USA.10.1109/eIT53891.2022.9813866 ↗
25. Drone-enabled bridge inspection methodology and application10.1016/j.autcon.2018.06.006 ↗
26. Developments, challenges, and perspectives of railway inspection robots10.1016/j.autcon.2022.104242 ↗
27. Flammini, F., Pragliola, C., and Smarra, G. (2017). Railway Infrastructure Monitoring by Drones, Institute of Electrical and Electronics Engineers Inc.10.1109/ESARS-ITEC.2016.7841398 ↗
28. Shi, L., Marcano, N.J.H., and Jacobsen, R.H. (2019, January 28–30). A Survey on Multi-unmanned Aerial Vehicle Communications for Autonomous Inspections. Proceedings of the 2019 22nd Euromicro Conference on Digital System Design (DSD), Kallithea, Greece.10.1109/DSD.2019.00088 ↗
29. Asaamoning, G., Mendes, P., Rosário, D., and Cerqueira, E. (2021). Drone Swarms as Networked Control Systems by Integration of Networking and Computing. Sensors, 21.10.3390/s21082642 ↗
30. Shi, L., Hernández Marcano, N.J., and Jacobsen, R.H. (2022, January 18–20). Multi-UAV Path Coordination through Generalized Potential Games. Proceedings of the 2022 8th International Conference on Automation, Robotics and Applications (ICARA), Prague, Czech Republic.10.1109/ICARA55094.2022.9738578 ↗
31. (2022, September 27). Avnet Ultra96-V2 Product Page. Available online: https://www.avnet.com/wps/portal/us/products/new-product-introductions/npi/aes-ultra96-v2/.
32. Nyboe, F.F., Malle, N.H., and Ebeid, E. (2022, January 21–24). MPSoC4Drones: An Open Framework for ROS2, PX4, and FPGA Integration. Proceedings of the 2022 International Conference on Unmanned Aircraft Systems (ICUAS), Dubrovnik, Croatia.10.1109/ICUAS54217.2022.9836055 ↗
33. Malle, N.H., Nyboe, F.F., and Ebeid, E. (2021, January 15–18). Survey and Evaluation of Sensors for Overhead Cable Detection using UAVs. Proceedings of the nternational Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece.10.1109/ICUAS51884.2021.9476724 ↗
34. Matlekovic, L., and Schneider-Kamp, P. (2022, January 26–27). From Monolith to Microservices: Software Architecture for Autonomous UAV Infrastructure Inspection. Proceedings of the 11th International Conference on Cloud Computing: Services and Architecture (CLOUD 2022), Vienna, Austria.10.5121/csit.2022.120622 ↗
35. Microservices for autonomous UAV inspection with UAV simulation as a service10.1016/j.simpat.2022.102548 ↗
36. Wireless Communications for Internet of Farming: An Early 5G Measurement Study10.1109/ACCESS.2022.3211096 ↗
37. A survey of formation control and motion planning of multiple unmanned vehicles10.1017/S0263574718000218 ↗
38. Semtech (2022, August 22). Long Range, Low Power 2.4 GHz Wireless RF Transceiver with Ranging Capability. Available online: https://www.semtech.com/products/wireless-rf/24-ghz-transceivers/sx1280.
39. Morgenthaler, S., Braun, T., Zhao, Z., Staub, T., and Anwander, M. (2012, January 3–7). UAVNet: A mobile wireless mesh network using Unmanned Aerial Vehicles. Proceedings of the 2012 IEEE Globecom Workshops, Anaheim, CA, USA.10.1109/GLOCOMW.2012.6477825 ↗
40. Pojda, J., Wolff, A., Sbeiti, M., and Wietfeld, C. (2011, January 6–9). Performance analysis of mesh routing protocols for UAV swarming applications. Proceedings of the 2011 8th International Symposium on Wireless Communication Systems, Aachen, Germany.10.1109/ISWCS.2011.6125375 ↗
41. Yanmaz, E., Hayat, S., Scherer, J., and Bettstetter, C. (2014, January 6–9). Experimental performance analysis of two-hop aerial 802.11 networks. In Proceedings of the 2014 IEEE Wireless Communications and Networking Conference (WCNC), Istanbul, Turkey.10.1109/WCNC.2014.6953010 ↗
42. (2012). ISO/IEC/IEEE International Standard—Information Technology–Telecommunications and Information Exchange between Systems Local and Metropolitan Area Networks–Specific Requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications (Standard No. ISO/IEC/IEEE 8802-11:2012(E) (Revision of ISO/IEC/IEEE 8802-11-2005 and Amendments)).
43. ROS.org (2022, October 10). Rosbridge_SUITE. Available online: http://wiki.ros.org/rosbridge_suite.
44. Onboard Powerline Perception System for UAVs Using mmWave Radar and FPGA-Accelerated Vision10.1109/ACCESS.2022.3217537 ↗
45. Google (2022, October 10). OR-Tools. Available online: https://developers.google.com/optimization.
46. Shi, L., Mehrooz, G., and Jacobsen, R.H. (2021, January 15–18). Inspection Path Planning for Aerial Vehicles via Sampling-based Sequential Optimization. Proceedings of the 2021 International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece.10.1109/ICUAS51884.2021.9476784 ↗
47. Zhou, B., Gao, F., Pan, J., and Shen, S. (August, January 31). Robust Real-time UAV Replanning Using Guided Gradient-based Optimization and Topological Paths. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.10.1109/ICRA40945.2020.9196996 ↗
48. Google (2022, October 10). Fast-Planner. Available online: https://github.com/HKUST-Aerial-Robotics/Fast-Planner.
49. Time-Varying Formation Tracking Control for Unmanned Aerial Vehicles: Theories and Applications10.1016/j.ifacol.2022.05.009 ↗
50. Leader–follower formation control of nonholonomic mobile robots with input constraints10.1016/j.automatica.2007.09.019 ↗
51. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
52. Chen, X., Tang, J., and Lao, S. (2020). Review of Unmanned Aerial Vehicle Swarm Communication Architectures and Routing Protocols. Appl. Sci., 10.10.3390/app10103661 ↗
53. Kim, Y., Perrig, A., and Tsudik, G. (2000, January 1–4). Simple and Fault-Tolerant Key Agreement for Dynamic Collaborative Groups. Proceedings of the 7th ACM Conference on Computer and Communications Security (CCS’00), Athens, Greece.10.1145/352600.352638 ↗
54. Maruyama, Y., Kato, S., and Azumi, T. (2016, January 1–7). Exploring the Performance of ROS2. Proceedings of the 13th International Conference on Embedded Software (EMSOFT’16), Pittsburgh, PA, USA.10.1145/2968478.2968502 ↗
55. Hansen, S. (2020). Modelling Low-Power Wide-Area Network for Autonomous Drone Systems. [Master’s Thesis, Aarhus University].
56. Ayoub, N., and Schneider-Kamp, P. (2021). Real-Time On-Board Deep Learning Fault Detection for Autonomous UAV Inspections. Electronics, 10.10.3390/electronics10091091 ↗
57. Ayoub, N., and Schneider-Kamp, P. (2020, January 8–10). Real-time on-board detection of components and faults in an autonomous uav system for power line inspection. Proceedings of the 1st International Conference on Deep Learning Theory and Applications, DeLTA 2020, Paris, France.10.5220/0009826700680075 ↗