Computer Vision in Autonomous Unmanned Aerial Vehicles—A Systematic Mapping Study
Source abstract
Abstract
Personal assistant robots provide novel technological solutions in order to monitor people’s activities, helping them in their daily lives. In this sense, unmanned aerial vehicles (UAVs) can also bring forward a present and future model of assistant robots. To develop aerial assistants, it is necessary to address the issue of autonomous navigation based on visual cues. Indeed, navigating autonomously is still a challenge in which computer vision technologies tend to play an outstanding role. Thus, the design of vision systems and algorithms for autonomous UAV navigation and flight control has become a prominent research field in the last few years. In this paper, a systematic mapping study is carried out in order to obtain a general view of this subject. The study provides an extensive analysis of papers that address computer vision as regards the following autonomous UAV vision-based tasks: (1) navigation, (2) control, (3) tracking or guidance, and (4) sense-and-avoid. The works considered in the mapping study—a total of 144 papers from an initial set of 2081—have been classified under the four categories above. Moreover, type of UAV, features of the vision systems employed and validation procedures are also analyzed. The results obtained make it possible to draw conclusions about the research focuses, which UAV platforms are mostly used in each category, which vision systems are most frequently employed, and which types of tests are usually performed to validate the proposed solutions. The results of this systematic mapping study demonstrate the scientific community’s growing interest in the development of vision-based solutions for autonomous UAVs. Moreover, they will make it possible to study the feasibility and characteristics of future UAVs taking the role of personal assistants.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotics and Sensor-Based Localization · Robotic Path Planning Algorithms · Aerospace Engineering and Control Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Escuela Técnica Superior de Ingenieros Industriales, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
Departamento de Sistemas Informáticos, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
Departamento de Ingeniería Eléctrica, Electrónica, Automática y Comunicaciones, Universidad de Castilla-La Mancha, 02071 Albacete, Spain
Funding
Agencia Estatal de Investigación
References (100+)
Show source-provided references
1. González-Jorge, H., Martínez-Sánchez, J., Bueno, M., and Arias, P. (2017). Unmanned Aerial Systems for Civil Applications: A Review. Drones, 1.10.3390/drones1010002 ↗
2. Aire (2019, March 18). A Self-Flying Robotic Assistant for the Home by Aevena Aire. Available online: https://esist.tech/2017/09/21/aire-a-self-flying-robotic-assistant-for-the-home-by-aevena-aire/.
3. Fleye (2019, March 18). Fleye—Your Personal Flying Robot. Available online: https://www.kickstarter.com/projects/gofleye/fleye-your-personal-flying-robot.
4. CIMON (2019, March 18). Assisting Astronauts with Airbus Innovation. Available online: https://www.airbus.com/newsroom/stories/Assisting-astronauts-with-Airbus-innovation.html.
5. Bualat, M.G., Smith, T., Smith, E.E., Fong, T., and Wheeler, D. (June, January 28). Astrobee: A New Tool for ISS Operations. Proceedings of the 15th International Conference on Space Operations. American Institute of Aeronautics and Astronautics, Marseille, France.
6. Belmonte, L.M., Morales, R., García, A.S., Segura, E., Novais, P., and Fernández-Caballero, A. (2019, January 26–28). Assisting Dependent People at Home through Autonomous Unmanned Aerial Vehicles. Proceedings of the ISAmI 2019, Ávila, Spain.10.1007/978-3-030-24097-4_26 ↗
7. Belmonte, L.M., Morales, R., García, A.S., Segura, E., Novais, P., and Fernández-Caballero, A. (2019, January 3–7). Trajectory Planning of a Quadrotor to Monitor Dependent People. Proceedings of the IWINAC 2019, Almería, Spain.10.1007/978-3-030-19591-5_22 ↗
8. Usability evaluation methods for the web: A systematic mapping study10.1016/j.infsof.2011.02.007 ↗
9. Tracking for mobile devices: A systematic mapping study10.1016/j.cag.2016.02.002 ↗
10. A systematic mapping study about socio-technical congruence10.1016/j.infsof.2017.10.004 ↗
11. Moguel, E., Conejero, J.M., Sánchez-Figueroa, F., Hernández, J., Preciado, J.C., and Rodríguez-Echeverría, R. (2017). Towards the Use of Unmanned Aerial Systems for Providing Sustainable Services in Smart Cities. Sensors, 18.10.3390/s18010064 ↗
12. A Taxonomy of Vision Systems for Ground Mobile Robots10.5772/58900 ↗
13. Survey on Computer Vision for UAVs: Current Developments and Trends10.1007/s10846-017-0483-z ↗
14. Survey of computer vision algorithms and applications for unmanned aerial vehicles10.1016/j.eswa.2017.09.033 ↗
15. A survey on vision-based UAV navigation10.1080/10095020.2017.1420509 ↗
16. RG, E. (2018, January 10). Template for a Mapping Study. Available online: https://community.dur.ac.uk/ebse/resources/templates/MappingStudyTemplate.pdf.
17. Munguía, R., Urzua, S., Bolea, Y., and Grau, A. (2016). Vision-based SLAM system for unmanned aerial vehicles. Sensors, 16.10.3390/s16030372 ↗
18. Vision-aided estimation of attitude, velocity, and inertial measurement bias for UAV stabilization10.1007/s10846-015-0206-2 ↗
19. Vision-based state estimation of an unmanned aerial vehicle10.3923/tb.2017.11.19 ↗
20. Visual based landing for an unmanned quadrotor10.1007/s10846-015-0271-6 ↗
21. Vision based displacement detection for stabilized UAV control on cloud server
22. Adaptive visual servoing of unmanned aerial vehicles in GPS-denied environments10.1109/TMECH.2017.2755669 ↗
23. Robust visual detection-learning-tracking framework for autonomous aerial refueling of UAVs10.1109/TIM.2015.2509318 ↗
24. Fuzzy logic controller for predictive vision-based target tracking with an unmanned aerial vehicle10.1080/01691864.2016.1271500 ↗
25. A novel trail detection and scene understanding framework for a quadrotor UAV with monocular vision10.1109/JSEN.2017.2746184 ↗
26. Wan, Y., Tang, J., and Lao, S. (2019). Research on the collision avoidance algorithm for fixed-wing UAVs based on maneuver coordination and planned trajectories prediction. Appl. Sci., 9.10.3390/app9040798 ↗
27. Sky region obstacle detection and tracking for vision-based UAS sense and avoid10.1007/s10846-015-0285-0 ↗
28. Collision avoidance for quadrotor using stereo vision depth maps10.1109/TAES.2015.140222 ↗
29. Al-Kaff, A., García, F., Martín, D., De La Escalera, A., and Armingol, J. (2017). Obstacle detection and avoidance system based on monocular camera and size expansion algorithm for UAVs. Sensors, 17.10.3390/s17051061 ↗
30. Vision-based terrain referenced navigation for unmanned aerial vehicles using homography relationship10.1007/s10846-012-9750-1 ↗
31. Monocular vision system for fixed altitude flight of unmanned aerial vehicles10.3390/s150716848 ↗
32. Dynamic visual servoing with image moments for an unmanned aerial vehicle using a virtual spring approach10.1080/01691864.2013.776967 ↗
33. Vision-based tracking and motion estimation for moving targets using unmanned air vehicles10.2514/1.33206 ↗
34. Vision-based navigation of unmanned aerial vehicles10.1016/j.conengprac.2010.03.004 ↗
35. Autonomous vision-based micro air vehicle for indoor and outdoor navigation10.1002/rob.21506 ↗
36. García Carrillo, L.R., Dzul López, A.E., Lozano, R., and Pégard, C. (2013). Quad Rotorcraft Control, Springer. Advances in Industrial Control.10.1007/978-1-4471-4399-4 ↗
37. Lozano, R. (2013). Unmanned Aerial Vehicles. Embedded Control, John Wiley & Sons, Inc.10.1002/9781118599938 ↗
38. Stereo vision guiding for the autonomous landing of fixed-wing UAVs: A saliency-inspired approach10.5772/62257 ↗
39. Reactive collision avoidance of unmanned aerial vehicles using a single vision sensor10.2514/1.57131 ↗
40. A vision-based automatic landing method for fixed-wing UAVs10.1007/s10846-009-9382-2 ↗
41. Vision algorithms for fixed-wing unmanned aerial vehicle landing system10.1007/s11431-016-0618-3 ↗
42. A binocular vision-based UAVs autonomous aerial refueling platform10.1007/s11432-016-5553-5 ↗
43. Feature article: Vision-based UAV collision avoidance with 2D dynamic safety envelope10.1109/MAES.2016.150155 ↗
44. Robust pose estimation for multirotor UAVs using off-board monocular vision10.1109/TIE.2017.2696482 ↗
45. Output feedback observation and control for visual servoing of VTOL UAVs10.1002/rnc.1638 ↗
46. Generalized Proportional Integral Control for an Unmanned Quadrotor System10.5772/60833 ↗
47. Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Toledo-Moreo, F.J., and Adeli, H. (2015). Robust Linear Longitudinal Feedback Control of a Flapping Wing Micro Air Vehicle. Artificial Computation in Biology and Medicine, Springer.10.1007/978-3-319-18914-7 ↗
48. Analysis and Fabrication of Unconventional Flapping Wing Air Vehicles10.1260/1756-8293.7.1.71 ↗
49. Rongfa, M.N., Pantuphag, T., and Srigrarom, S. (2016). Analysis of Kinematics of Flapping Wing UAV Using OptiTrack Systems. Aerospace, 3.10.3390/aerospace3030023 ↗
50. Grossman, D. (2019, March 18). Airships, Dirigibles, Zeppelins & Blimps: What’s the Difference?. Available online: https://www.airships.net/dirigible/.
51. Unmanned aerial vehicle localization based on monocular vision and online mosaicking10.1007/s10846-008-9305-7 ↗
52. The Autonomous Blimp Project of LAAS-CNRS: Achievements in Flight Control and Terrain Mapping10.1177/0278364904042200 ↗
53. Vision-based monitoring of locally linear structures using an unmanned aerial vehicle10.1061/(ASCE)1076-0342(2008)14:1(52) ↗
54. Vision-based unmanned aerial vehicle navigation using geo-referenced information10.1155/2009/387308 ↗
55. Holness, A.E., Bruck, H.A., and Gupta, S.K. (2018). Characterizing and modeling the enhancement of lift and payload capacity resulting from thrust augmentation in a propeller-assisted flapping wing air vehicle. Int. J. Micro Air Veh., 50–60.10.1177/1756829317734836 ↗
56. Mohamed, A., Novais, P., Pereira, A., Villarrubia, G., and Fernández-Caballero, A. (2015). Flying Depth Camera for Indoor Mapping and Localization. Ambient Intelligence—Software and Applications, Springer.10.1007/978-3-319-19695-4 ↗
57. Visual measurement in simulation environment for vision-based UAV autonomous aerial refueling10.1109/TIM.2014.2343392 ↗
58. Monocular vision-based real-time target recognition and tracking for autonomously landing an UAV in a cluttered shipboard environment10.1007/s10514-016-9564-2 ↗
59. Image-based visual servoing of a quadrotor using virtual camera approach10.1109/TMECH.2016.2639531 ↗
60. Vision-based autonomous hovering for a miniature quad-rotor10.1017/S0263574713000611 ↗
61. Control of a quadrotor helicopter using dual camera visual feedback10.1177/0278364905053804 ↗
62. Multi-resolution visual fiducial and assistant navigation system for unmanned aerial vehicle landing10.1016/j.ast.2017.03.008 ↗
63. Verification of monocular and binocular pose estimation algorithms in vision-based UAVs autonomous aerial refueling system10.1007/s11431-016-6097-z ↗
64. Ground stereo vision-based navigation for autonomous take-off and landing of UAVs: A Chan-Vese model approach10.5772/62027 ↗
65. Long-range stereo visual odometry for extended altitude flight of unmanned aerial vehicles10.1177/0278364915581194 ↗
66. Kong, W., Hu, T., Zhang, D., Shen, L., and Zhang, J. (2017). Localization framework for real-time UAV autonomous landing: An on-ground deployed visual approach. Sensors, 17.10.3390/s17061437 ↗
67. Hein, D., Kraft, T., Brauchle, J., and Berger, R. (2019). Integrated UAV-Based Real-Time Mapping for Security Applications. ISPRS Int. J. Geo-Inf., 8.10.3390/ijgi8050219 ↗
68. Multi-camera tracking and mapping for unmanned aerial vehicles in unstructured environments10.1007/s10846-014-0085-y ↗
69. Combining stereo vision and inertial navigation system for a quad-rotor UAV10.1007/s10846-011-9571-7 ↗
70. On-Board Dual-Stereo-Vision for the Navigation of an Autonomous MAV10.1007/s10846-013-9907-6 ↗
71. Monocular visual-inertial SLAM-based collision avoidance strategy for fail-safe UAV using fuzzy logic controllers10.1007/s10846-013-9918-3 ↗
72. Hinas, A., Roberts, J., and Gonzalez, F. (2017). Vision-based target finding and inspection of a ground target using a multirotor UAV system. Sensors, 17.10.3390/s17122929 ↗
73. A visual global positioning system for unmanned aerial vehicles used in photogrammetric applications10.1007/s10846-010-9489-5 ↗
74. Recognition of a landing platform for unmanned aerial vehicles by using computer vision-based techniques10.1016/j.eswa.2017.01.017 ↗
75. Detection of aircraft below the horizon for vision-based detect and avoid in unmanned aircraft systems10.1002/rob.21719 ↗
76. A Tandem Active Disturbance Rejection Control for a Laboratory Helicopter With Variable-Speed Rotors10.1109/TIE.2016.2587238 ↗
77. Belmonte, L.M., Morales, R., Fernández-Caballero, A., and Somolinos, J. (2016). Robust Decentralized Nonlinear Control for a Twin Rotor MIMO System. Sensors, 16.10.5772/64875 ↗
78. Belmonte, L.M., Morales, R., Fernández-Caballero, A., and Somolinos, J.A. (2016). Nonlinear Cascade-Based Control for a Twin Rotor MIMO System. Nonlinear Systems—Design, Analysis, Estimation and Control, InTech.10.5772/64875 ↗
79. Siti, I., Mjahed, M., Ayad, H., and El Kari, A. (2019). New trajectory tracking approach for a quadcopter using genetic algorithm and reference model methods. Appl. Sci., 9.10.3390/app9091780 ↗
80. Visual-based quadrotor control by means of fuzzy cognitive maps10.1016/j.isatra.2015.11.007 ↗
81. Quadrotor circumnavigation of an unknown moving target using camera vision-based measurements10.1049/iet-cta.2015.1246 ↗
82. Landing site searching and selection algorithm development using vision system and its application to quadrotor10.1109/TCST.2014.2341214 ↗
83. Autonomous vision-based helicopter flights through obstacle gates10.1007/s10846-009-9357-3 ↗
84. Visual 3D SLAM from UAVs10.1007/s10846-008-9304-8 ↗
85. An IMU/UWB/vision-based extended Kalman filter for mini-UAV localization in indoor environment using 802.15.4a wireless sensor network10.1007/s10846-012-9742-1 ↗
86. Vision-based odometry and SLAM for medium and high altitude flying UAVs10.1007/s10846-008-9257-y ↗
87. Computer vision onboard UAVs for civilian tasks10.1007/s10846-008-9256-z ↗
88. Image based and hybrid visual servo control of an unmanned aerial vehicle10.1007/s10846-011-9582-4 ↗
89. A vision-based guidance system for UAV navigation and safe landing using natural landmarks10.1007/s10846-009-9373-3 ↗
90. Airborne vision-based navigation method for UAV accuracy landing using infrared lamps10.1007/s10846-013-9819-5 ↗
91. Monocular visual mapping for obstacle avoidance on UAVs10.1007/s10846-013-9967-7 ↗
92. A hierarchical tracking strategy for vision-based applications on-board UAVs10.1007/s10846-013-9814-x ↗
93. Omnidirectional vision for UAV: Applications to attitude, motion and altitude estimation for day and night conditions10.1007/s10846-012-9752-z ↗
94. Stability analysis of a vision-based UAV controller10.1007/s10846-013-9946-z ↗
95. EKF based attitude estimation and stabilization of a quadrotor UAV using vanishing points in catadioptric images10.1007/s10846-010-9459-y ↗
96. Simulation environment for machine vision based aerial refueling for UAVs10.1109/TAES.2009.4805269 ↗
97. Hovering quad-rotor control: A comparison of nonlinear controllers using visual feedback10.1109/TAES.2012.6324687 ↗
98. Vision-based sense-and-avoid framework for unmanned aerial vehicles10.1109/TAES.2015.140252 ↗
99. Adaptive visual servoing of UAVs using a virtual camera10.1109/TAES.2016.15-0155 ↗
100. Novel technique for vision-based UAV navigation10.1109/TAES.2011.6034661 ↗