Netting and Background Distinction with In-flight Optic Flow
Source abstract
Abstract
Web-like obstacles, such as safety nets, represent a unique hazard for drones, and especially UAVs (Unmanned Aerial Vehicles). Fencing and netting are particularly difficult to distinguish from the background using either computer vision, lidar and sonar. In contrast, animals such as flying insects may detect these web-like obstacles using Optic Flow (OF), and more precisely motion parallax. A netting-avoidance solution was proposed using a OF-based detection method. The nettingdetection method was based on a signature defined by the shape of the OF magnitude across the visual field. We established that the OF shape depends on the orientation of the netting in relation to the hexarotor’s movement.This paper demonstrates netting detection in real-world experiments, according to any direction flight made by the UAV along the net. The proposed NOWA method (which stands for Netting Optical-floW-based distinction Algorithm) separates the OF signatures belonging to these different surfaces -netting or background- whatever their orientations. By extracting the OF signatures of these different surfaces and separating them, the proposed visual method can estimate their relative locations and orientations. In a robotic simulations, the multirotor explores and navigates automatically using this netting detection method, using saccades to avoid obstacles. In the simulations, these saccades are also used to simplify netting detection by orienting itself systematically parallel to these planes, a behavior reminiscent of flying insects.
Source: openaire · Quality: candidate unverified · Source record ↗
Research context
OpenAlex topics: Advanced Vision and Imaging · Advanced Optical Sensing Technologies · Infrared Target Detection Methodologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (36)
Show source-provided references
1. Ali, H., Seifert, C., Jindal, N., Paletta, L., & Paar, G. (2007). Window detection in facades. 14th International Conference on Image Analysis and Processing (ICIAP 2007) (pp. 837–842). China: Beijing. https://doi.org/10.1109/WRC-SARA.2018.858421310.1109/WRC-SARA.2018.8584213 ↗
2. Tibebu, H., Roche, J., De Silva, V., & Kondoz, A. (2021). Lidar-based glass detection for improved occupancy grid mapping. Sensors, 21(7), 2263. https://doi.org/10.3390/s2107226310.3390/s21072263 ↗
3. Zhao, X., Yang, Z., & Schwertfeger, S. (2020). Mapping with reflection-detection and utilization of reflection in 3D LiDAR scans. In: 2020 IEEE InternationalSymposium on Safety, Security, and Rescue Robotics (SSRR), Abu Dhabi, United Arab Emirates, pp. 27–33. https://doi.org/10.1109/SSRR50563.2020.929259510.1109/SSRR50563.2020.9292595 ↗
4. Weerakoon, L., Herr, G.S., Blunt, J., Yu, M., & Chopra, N. (2022). Cartographer_glass: 2D graph SLAM framework using LiDAR for glass environments. arXiv preprint arXiv:2212.08633. https://doi.org/10.48550/arXiv.2212.0863310.48550/arXiv.2212.08633 ↗
5. Wang, R., Bach, J., & Ferrie, F.P. (2011). Window detection from mobile LiDAR data. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV), Kona, HI, USA, pp. 58–65. https://doi.org/10.1109/WACV.2011.571148410.1109/WACV.2011.5711484 ↗
6. Wei, H., Li, X.-E., Shi, Y., You, B., & Xu, Y. (2018). Multi-sensor fusion glass detection for robot navigation and mapping. 2018 WRC Symposium on Advanced Robotics and Automation (WRC SARA) (pp. 184–188). China: Beijing. https://doi.org/10.1109/WRC-SARA.2018.858421310.1109/WRC-SARA.2018.8584213 ↗
7. Kern, R., Boeddeker, N., Dittmar, L., & Egelhaaf, M. (2012). Blowfly flight characteristics are shaped by environmental features and controlled by optic flow information. Journal of Experimental Biology, 215(14), 2501–2514. https://doi.org/10.1242/jeb.06171310.1242/jeb.061713 ↗
8. Sarmiento, T. (2018). Insights on obstacle avoidance for small unmanned aerial systems from a study of flying animal behavior. Robotics and Autonomous Systems, 99, 17–29. https://doi.org/10.1016/j.robot.2017.09.0010.1016/j.robot.2017.09.00 ↗
9. Sathyakumar, V. (2022). How do stingless bees navigate around spider webs? Thesis, Faculty of Science of Engineering, Macquarie University, Sydney, Australia. https://doi.org/10.25949/21539952.v110.25949/21539952.v1 ↗
10. Bruce, M., Heiling, A., & Herberstein, M. (2005). Spider signals: Are web decorations visible to birds and bees? Biology letters, 1, 299–302. https://doi.org/10.1098/rsbl.2005.030710.1098/rsbl.2005.0307 ↗
11. Franceschini, N., Pichon, J. M., Blanes, C., & Brady, J. M. (1992). From insect vision to robot vision [and discussion]. Philosophical Transactions: Biological Sciences, 337(1281), 283–294. https://doi.org/10.1098/rstb.1992.010610.1098/rstb.1992.0106 ↗
12. Ruffier, F., & Franceschini, N. (2005). Optic flow regulation: the key to aircraft automatic guidance. Robotics and Autonomous Systems, 50(4), 177–194. https://doi.org/10.1016/j.robot.2004.09.01610.1016/j.robot.2004.09.016 ↗
13. Expert, F., & Ruffier, F. (2015). Flying over uneven moving terrain based on optic-flow cues without any need for reference frames or accelerometers. Bioinspiration & Biomimetics, 10(2), 026003. https://doi.org/10.1088/1748-3182/10/2/02600310.1088/1748-3182/10/2/026003 ↗
14. Ho, H. W., de Croon, G. C., Van Kampen, E., Chu, Q., & Mulder, M. (2018). Adaptive gain control strategy for constant optical flow divergence landing. IEEE Transactions on Robotics, 34(2), 508–516. https://doi.org/10.1109/TRO.2018.281741810.1109/TRO.2018.2817418 ↗
15. De Croon, G. C., Dupeyroux, J. J., De Wagter, C., Chatterjee, A., Olejnik, D. A., & Ruffier, F. (2022). Accommodating unobservability to control flight attitude with optic flow. Nature, 610(7932), 485–490. https://doi.org/10.1038/s41586-022-05182-210.1038/s41586-022-05182-2 ↗
16. Serres, J. R., & Ruffier, F. (2017). Optic flow-based collision-free strategies: From insects to robots. Arthropod Structure & Development, 46(5), 703–717. https://doi.org/10.1016/j.asd.2017.06.00310.1016/j.asd.2017.06.003 ↗
17. Hérissé, B., Hamel, T., Mahony, R., & Russotto, F.-X. (2010). A terrain-following control approach for a vtol unmanned aerial vehicle using average optical flow. Autonomous Robots, 29, 381–399. https://doi.org/10.1007/s10514-010-9208-x10.1007/s10514-010-9208-x ↗
18. Vogel, C., Roth, S., & Schindler, K. (2013). Piecewise rigid scene flow. In: 2013 International Conference on Computer Vision. Institute of Electrical and Electronics Engineers, Sydney, NSW, Australia. https://doi.org/10.1109/ICCV.2013.17410.1109/ICCV.2013.174 ↗
19. Lucas, B. D., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. IJCAI’81: 7th International Joint Conference on Artificial Intelligence 1981 (Vol. 2, pp. 674–679), Canada, Vancouver.
20. Jianbo, S., & Tomasi, C. (1994). Good features to track. In: 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 593–600. https://doi.org/10.1109/CVPR.1994.32379410.1109/CVPR.1994.323794 ↗
21. Farnebäck, G. (2003). Two-frame motion estimation based on polynomial expansion. Proceedings of the 13th Scandinavian conference on Image analysis, Berlin, Germany, pp. 363–370. https://doi.org/10.1007/3-540-45103-X_5010.1007/3-540-45103-X_50 ↗
22. Adarve, J. D., & Mahony, R. (2016). A filter formulation for computing real time optical flow. IEEE Robotics and Automation Letters, 1(2), 1192–1199. https://doi.org/10.1109/LRA.2016.253292810.1109/LRA.2016.2532928 ↗
23. Koenderink, J. J., & van Doorn, A. J. (1987). Facts on optic flow. Biological Cybernetics, 56(4), 247–254. https://doi.org/10.1007/BF0036521910.1007/BF00365219 ↗
24. Daini, X., Coquet, C., Raffin, R., Raharijaona, T., & Ruffier, F. (2023). Safety net detection by optic flow processing. In: 2023 International Conference on Unmanned Aircraft Systems (ICUAS), Warsaw, Poland, pp. 32–39. https://doi.org/10.1109/ICUAS57906.2023.1015659710.1109/ICUAS57906.2023.10156597 ↗
25. Corke, P. (2017). Robotics. Vision and Control. Springer. https://doi.org/10.1007/978-3-319-54413-710.1007/978-3-319-54413-7 ↗
26. Vassallo, R.F., Santos-Victor, J., & Schneebeli, H.J. (2002). A general approach for egomotion estimation with omnidirectional images. In: Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in Conjunction with ECCV’02, Seoul, Korea (South), pp. 97–103. https://doi.org/10.1109/OMNVIS.2002.104450210.1109/OMNVIS.2002.1044502 ↗
27. Gluckman, J., & Nayar, S.K. (1998). Ego-motion and omnidirectional cameras. In: Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271), Bombay, India, pp. 999–1005. https://doi.org/10.1109/ICCV.1998.71083810.1109/ICCV.1998.710838 ↗
28. Vassallo, R.F., Santos-Victor, J., & Schneebeli, H.J. (2002). A general approach for egomotion estimation with omnidirectional images. In: Proceedings of the IEEEWorkshop on Omnidirectional Vision 2002. Held in Conjunction with ECCV’02, Seoul, Korea (South), pp. 97–103. https://doi.org/10.1109/OMNVIS.2002.104450210.1109/OMNVIS.2002.1044502 ↗
29. Castillo-Zamora, J.J., Escareno, J., Boussaada, I., Labbani, O., & Camarillo, K. (2019). Modeling and control of an aerial multi-cargo system: Robust acquiring and transport operations. In: 2019 18thEuropean Control Conference (ECC), Naples, Italy, pp. 1708–1713. https://doi.org/10.23919/ECC.2019.879604510.23919/ECC.2019.8796045 ↗
30. Daini, X.: Camera images from embedded Camera on a drone flying in front of walls and nettings in simulation and experimental data. https://entrepot.recherche.data.gouv.fr/dataset.xhtml?persistentId=https://doi.org/10.57745/DZIM2L(2024).10.57745/DZIM2L ↗
31. Eckmeier, D., Geurten, B. R., Kress, D., Mertes, M., Kern, R., Egelhaaf, M., & Bischof, H.-J. (2008). Gaze strategy in the free flying zebra finch (taeniopygia guttata). PLoS One, 3(12), 3956.10.1371/journal.pone.0003956 ↗
32. Ho, H. W., de Croon, G. C., & Chu, Q. (2017). Distance and velocity estimation using optical flow from a monocular camera. International Journal of Micro Air Vehicles, 9(3), 198–208. https://doi.org/10.1177/175682931769556610.1177/1756829317695566 ↗
33. Bergantin, L., Coquet, C., Dumon, J., Negre, A., Raharijaona, T., Marchand, N., & Ruffier, F. (2023). Indoor and outdoor in-flight odometry based solely on optic flows with oscillatory trajectories. International Journal of Micro Air Vehicles, 15, 17568293221148380. https://doi.org/10.1177/1756829322114838010.1177/17568293221148380 ↗
34. Roubieu, F. L., Serres, J. R., Colonnier, F., Franceschini, N., Viollet, S., & Ruffier, F. (2014). A biomimetic vision-based hovercraft accounts for bees’ complex behaviour in various corridors. Bioinspiration & Biomimetics, 9(3), 036003. https://doi.org/10.1088/1748-3182/9/3/03600310.1088/1748-3182/9/3/036003 ↗
35. Serres, J. R., & Ruffier, F. (2015). Biomimetic autopilot based on minimalistic motion vision for navigating along corridors comprising u-shaped and s-shaped turns. Journal of Bionic Engineering, 12(1), 47–60. https://doi.org/10.1016/S1672-6529(14)60099-810.1016/S1672-6529(14)60099-8 ↗
36. Castillo-Zamora, J.J., Bergantin, L., & Ruffier, F. (2022). Corridor 3d navigation of a fully-actuated multirotor by means of bee-inspired optic flow regulation. In: 2022 26thInternational Conference on System Theory, Control and Computing (ICSTCC), Sinaia, Romania, pp. 318–324. https://doi.org/10.1109/ICSTCC55426.2022.993180910.1109/ICSTCC55426.2022.9931809 ↗