journal article·2023-11-09Open access

Vision-Guided Hierarchical Control and Autonomous Positioning for Aerial Manipulator

Xia Ye, Haohao Cui, Lidong Wang, Shangjun Xie, Hong Ni

Applied Sciences· Volume 13· Issue 22

Abstract

Aerial manipulator systems possess active operational capability, and by incorporating various sensors, the systems’ autonomy is further enhanced. In this paper, we address the challenge of accurate positioning between an aerial manipulator and the operational targets during tasks such as grasping and delivery in the absence of motion capture systems indoors. We propose a vision-guided aerial manipulator system comprising a quad-rotor UAV and a single-degree-of-freedom manipulator. First, the overall structure of the aerial manipulator is designed, and a hierarchical control system is established. We employ the fusion of LiDAR-based SLAM (simultaneous localization and mapping) and IMU (inertial measurement unit) to enhance the positioning accuracy of the aerial manipulator. Real-time target detection and recognition are achieved by combining a depth camera and laser sensor for distance measurements, enabling adjustment of the grasping pose of the aerial manipulator. Finally, we employ a segmented grasping strategy to position and grasp the target object precisely. Experimental results demonstrate that the designed aerial manipulator system maintains a stable orientation within a certain range of ±5° during operation; its position movement is independent of orientation changes. The successful autonomous grasping of lightweight cylindrical objects in real-world scenarios verifies the effectiveness and rationality of the proposed system, ensuring high operational efficiency and robust disturbance resistance.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Robotics and Sensor-Based Localization · Robotic Path Planning Algorithms · UAV Applications and Optimization

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Engineering, Hangzhou Normal University, Hangzhou 310018, China

Funding

National College Student Innovation and Entrepreneurship Training Program · General Projects of Zhejiang Provincial Department of Education

References (42)

Show source-provided references
1. Zhang, G., He, Y., Dai, B., Gu, F., Yang, L., Han, J., and Liu, G. (2019). Aerial Grasping of an Object in the Strong Wind: Robust Control of an Aerial Manipulator. Appl. Sci., 9.10.3390/app9112230 ↗
2. Verma, V., Gupta, D., Gupta, S., Uppal, M., Anand, D., Ortega-Mansilla, A., Alharithi, F.S., Almotiri, J., and Goyal, N. (2022). A Deep Learning-Based Intelligent Garbage Detection System Using an Unmanned Aerial Vehicle. Symmetry, 14.10.3390/sym14050960 ↗
3. Dhwani Fire: Aerial System for Extinguishing Fire10.1149/10701.10295ecst ↗
4. Unmanned Aerial Vehicles (UAVs): A Survey on Civil Applications and Key Research Challenges10.1109/ACCESS.2019.2909530 ↗
5. Loor, S.J., Bejarano, A.R., Silva, F.M., and Andaluz, V.H. (2020, January 22–25). Construction and Control Aerial Manipulator Robot. Proceedings of the International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, Kitakyushu, Japan.10.1007/978-3-030-55789-8_11 ↗
6. Sensor installation and retrieval operations using an unmanned aerial manip-ulator10.1109/LRA.2019.2918448 ↗
7. A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms
8. Sajjadi, S., Bittick, J., Janabi-Sharifi, F., and Mantegh, I. (2023, January 6–9). A Robust and Adaptive Sensor Fusion Approach for Indoor UAV Localization. Proceedings of the 2023 International Conference on Unmanned Aircraft Systems (ICUAS), Warsaw, Poland.10.1109/ICUAS57906.2023.10156526 ↗
9. Meng, X., He, Y., Gu, F., Yang, L., Dai, B., Liu, Z., and Han, J. (2016, January 3–7). Design and implementation of rotor aerial manipulator system. Proceedings of the 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), Qingdao, China.10.1109/ROBIO.2016.7866400 ↗
10. Suarez, A., Soria, P.R., Heredia, G., Arrue, B.C., and Ollero, A. (2017, January 24–28). Anthropomorphic, compliant and lightweight dual arm system for aerial manipulation. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.10.1109/IROS.2017.8202266 ↗
11. Design, modeling, and control of an aerial manipulator for placement and retrieval of sensors in the environment10.1002/rob.21963 ↗
12. Yamada, T., Tsuji, T., Paul, H., Martinez, R.R., Ladig, R., and Shimonomura, K. (2022, January 11–15). Exploration of UAV Rope Handling and Flight in Narrow Space Strategies, using a Three-arm Aerial Manipulator System. Proceedings of the 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), Sapporo, Japan.10.1109/AIM52237.2022.9863293 ↗
13. Peng, R., Chen, X., and Lu, P. (October, January 27). A Motion decoupled Aerial Robotic Manipulator for Better Inspection. Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.10.1109/IROS51168.2021.9636765 ↗
14. Zhang, W., Liu, Q., Wang, M., Jia, J., Lyu, S., Guo, K., Yu, X., and Guo, L. (2021, January 15–18). Design of an Aerial Manipulator System Applied to Capture Missions. Proceedings of the 2021 International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece.10.1109/ICUAS51884.2021.9476743 ↗
15. Chaikalis, D., Khorrami, F., and Tzes, A. (2020, January 1–4). Adaptive Control Approaches for an Unmanned Aerial Manipulation System. Proceedings of the 2020 International Conference on Unmanned Aircraft Systems (ICUAS), Athens, Greece.10.1109/ICUAS48674.2020.9213920 ↗
16. Chaikalis, D., Evangeliou, N., Tzes, A., and Khorrami, F. (2023, January 13–16). Decentralized Leader-Follower Visual Cooperative Package Transportation using Unmanned Aerial Manipulators. Proceedings of the 2023 European Control Conference (ECC), Bucharest, Romania.10.23919/ECC57647.2023.10178351 ↗
17. Forceful Aerial Manipulation Based on an Aerial Robotic Chain: Hybrid Modeling and Control10.1109/LRA.2021.3064254 ↗
18. Nekoo, S.R., Suarez, A., Acosta, J., Heredia, G., and Ollero, A. (2023, January 6–9). Constrained Design Optimization of a Long-Reach Dual-Arm Aerial Manipulator for Maintenance Tasks. Proceedings of the 2023 International Conference on Unmanned Aircraft Systems (ICUAS), Warsaw, Poland.10.1109/ICUAS57906.2023.10156252 ↗
19. Natural Feature-based Visual Servoing for Grasping Target with an Aerial Manipulator10.1007/s42235-020-0017-4 ↗
20. Thomas, J., Loianno, G., Sreenath, K., and Kumar, V. (June, January 31). Toward image-based visual servoing for aerial grasping and perching. Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China.10.1109/ICRA.2014.6907149 ↗
21. Wu, H., Jiang, L., Liu, X., Li, J., Yang, Y., and Zhang, S. (2021, January 18–20). Intelligent Explosive Ordnance Disposal UAV System Based on Manipulator and Real-Time Object Detection. Proceedings of the 2021 4th International Conference on Intelligent Robotics and Control Engineering (IRCE), Lanzhou, China.10.1109/IRCE53649.2021.9570974 ↗
22. Zhiyang, Z., Hongyu, C., Xiaoqiang, Z., Yanhua, S., Zhiyuan, C., and Yin, L. (2022, January 23–25). Grasp Pose Estimation for Aerial Manipulator Based on Improved 6-DOF GraspNet. Proceedings of the 2022 5th International Conference on Pattern Recognition and Artificial Intelligence (PRAI), Chengdu, China.10.1109/PRAI55851.2022.9904159 ↗
23. Image-Based Visual Servoing of Unmanned Aerial Manipulators for Tracking and Grasping a Moving Target10.1109/TII.2022.3222688 ↗
24. Vision-Based Hierarchical Impedance Control of an Aerial Manipulator10.1109/TIE.2022.3203755 ↗
25. Grasp Planning and Visual Servoing for an Outdoors Aerial Dual Manipulator10.1016/j.eng.2019.11.003 ↗
26. Kim, S., Seo, H., and Kim, H.J. (2015, January 26–30). Operating an unknown drawer using an aerial manipulator. Proceedings of the IEEE International Conference on Robotics and Automation, Seattle, WA, USA.10.1109/ICRA.2015.7139968 ↗
27. Seo, H., Kim, S., and Kim, H.J. (June, January 29). Aerial grasping of cylindrical object using visual servoing based on stochastic model predictive control. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.10.1109/ICRA.2017.7989751 ↗
28. Guidance for Autonomous Aerial Manipulator Using Stereo Vision10.1007/s10846-019-01060-8 ↗
29. Contact force control of aerial manipulator systems
30. Modeling and Dynamic Center of Gravity Compensation Control of a Rotorcraft Flight Robot Arm
31. Thomas, J., Loianno, G., Daniilidis, K., and Kumar, V. (2016, January 17–21). The role of vision in perching and grasping for MAVs. Proceedings of the Micro- and Nanotechnology Sensors, Systems, and Applications VIII, Bellingham, WA, USA.10.1117/12.2224056 ↗
32. Mandal, A.K., Seo, J.-B., De, S., Poddar, A.K., and Rohde, U. (2023, January 20–23). A Novel Statistically-Aided Learning Framework for Precise Localization of UAVs. Proceedings of the 2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring), Florence, Italy.10.1109/VTC2023-Spring57618.2023.10200280 ↗
33. A solution to the simultaneous localization and map building (SLAM) problem10.1109/70.938381 ↗
34. Rauf, A., Muhamamd, W., Mehmood, Z., and Irshad, M.J. (2023, January 8–9). Simultaneous Localization and Mapping of UAV For Precision Agricultural Application. Proceedings of the 2023 International Conference on Energy, Power, Environment, Control, and Computing (ICEPECC), Gujrat, Pakistan.10.1109/ICEPECC57281.2023.10209500 ↗
35. Lin, J., and Zhang, F. (August, January 31). Loam livox: A fast, robust, high-precision LiDAR odometry and mapping package for LiDARs of small FoV. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.10.1109/ICRA40945.2020.9197440 ↗
36. Low-drift and real-time lidar odometry and mapping10.1007/s10514-016-9548-2 ↗
37. Comparing ICP variants on real-world datasets10.1007/s10514-013-9327-2 ↗
38. Vanli, O.A., and Taylor, C.N. (2020, January 6–9). Covariance Estimation for Factor Graph Based Bayesian Estimation. Proceedings of the IEEE 23rd International Con-ference on Information Fusion (FUSION), Rustenburg, South Africa.10.23919/FUSION45008.2020.9190223 ↗
39. Quaternion-based state-dependent differential Riccati equation for quadrotor drones: Regulation control problem in aerobatic flight10.1017/S0263574722000091 ↗
40. FEUI: Fusion Embedding for User Identification across social networks10.1007/s10489-021-02716-5 ↗
41. Modeling and simulation of quadrotor UAV with tilting rotors10.1007/s12555-015-0064-5 ↗
42. Calibration on Camera’s Intrinsic Parameters Based on Orthogonal Learning Neural Network and Vanishing Points10.1109/JSEN.2019.2957134 ↗