journal article·2023-10-15Open access

UAV Detection and Tracking in Urban Environments Using Passive Sensors: A Survey

Xiaochen Yan, Tingting Fu, Huaming Lin, Feng Xuan, Yi Huang, Yuchen Cao, Haoji Hu, Peng Liu

Applied Sciences· Volume 13· Issue 20

Abstract

Unmanned aerial vehicles (UAVs) have gained significant popularity across various domains, but their proliferation also raises concerns about security, public safety, and privacy. Consequently, the detection and tracking of UAVs have become crucial. Among the UAV-monitoring technologies, those suitable for urban Internet-of-Things (IoT) environments primarily include radio frequency (RF), acoustic, and visual technologies. In this article, we provide a comprehensive review of passive UAV surveillance technologies, encompassing RF-based, acoustic-based, and vision-based methods for UAV detection, localization, and tracking. Our research reveals that certain lightweight UAV depth detection models have been effectively downsized for deployment on edge devices, facilitating the integration of edge computing and deep learning. In the city-wide anti-UAV, the integration of numerous urban infrastructure monitoring facilities presents a challenge in achieving a centralized computing center due to the large volume of data. To address this, calculations can be performed on edge devices, enabling faster UAV detection. Currently, there is a wide range of anti-UAV systems that have been deployed in both commercial and military sectors to address the challenges posed by UAVs. In this article, we provide an overview of the existing military and commercial anti-UAV systems. Furthermore, we propose several suggestions for developing general-purpose UAV-monitoring systems tailored for urban environments. These suggestions encompass considering the specific requirements of the application scenario, integrating detection and tracking mechanisms with appropriate countermeasures, designing for scalability and modularity, and leveraging advanced data analytics and machine learning techniques. To promote further research in the field of UAV-monitoring systems, we have compiled publicly available datasets comprising visual, acoustic, and radio frequency data. These datasets can be employed to evaluate the effectiveness of various UAV-monitoring techniques and algorithms. All of the datasets mentioned are linked in the text or in the references. Most of these datasets have been validated in multiple studies, and researchers can find more specific information in the corresponding papers or documents. By presenting this comprehensive overview and providing valuable insights, we aim to advance the development of UAV surveillance technologies, address the challenges posed by UAV proliferation, and foster innovation in the field of UAV monitoring and security.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Video Surveillance and Tracking Methods · UAV Applications and Optimization · Indoor and Outdoor Localization Technologies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

HDU-ITMO Joint School, Hangzhou Dianzi University, Hangzhou 310018, China

School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China

Hangzhou Security and Technology Evaluation Center, Hangzhou 310020, China

College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China

Funding

Zhejiang Public Information Industry Co., Ltd.

References (100+)

Show source-provided references
1. Tian, S., Li, Y., Zhang, X., Zheng, L., Cheng, L., She, W., and Xie, W. (2023). Fast UAV path planning in urban environments based on three-step experience buffer sampling DDPG. Digit. Commun. Netw.10.1016/j.dcan.2023.02.016 ↗
2. An amateur drone surveillance system based on the cognitive Internet of Things10.1109/MCOM.2017.1700452 ↗
3. Performance analysis of legitimate UAV surveillance system with suspicious relay and anti-surveillance technology10.1016/j.dcan.2021.10.009 ↗
4. Lin, N., Tang, H., Zhao, L., Wan, S., Hawbani, A., and Guizani, M. (2023). A PDDQNLP Algorithm for Energy Efficient Computation Offloading in UAV-assisted MEC. IEEE Trans. Wirel. Commun.10.1109/TWC.2023.3266497 ↗
5. A review of 6G autonomous intelligent transportation systems: Mechanisms, applications and challenges10.1016/j.sysarc.2023.102929 ↗
6. Maximizing Sensor Lifetime via Multi-node Partial-Charging on Sensors
7. An Approximation Algorithm for the h-Hop Independently Submodular Maximization Problem and Its Applications10.1109/TNET.2022.3210825 ↗
8. Cheng, Z., Liwang, M., Chen, N., Huang, L., Guizani, N., and Du, X. (2022). Learning-based user association and dynamic resource allocation in multi-connectivity enabled unmanned aerial vehicle networks. Digit. Commun. Netw., in press.10.1016/j.dcan.2022.05.026 ↗
9. Machine learning applications in internet-of-drones: Systematic review, recent deployments, and open issues10.1145/3571728 ↗
10. A review of Urban Air Mobility-enabled Intelligent Transportation Systems: Mechanisms, applications and challenges10.1016/j.sysarc.2023.102902 ↗
11. Iftikhar, S., Asim, M., Zhang, Z., Muthanna, A., Chen, J., El-Affendi, M., Sedik, A., and Abd El-Latif, A.A. (2023). Target Detection and Recognition for Traffic Congestion in Smart Cities Using Deep Learning-Enabled UAVs: A Review and Analysis. Appl. Sci., 13.10.3390/app13063995 ↗
12. Shi, J., Cong, P., Zhao, L., Wang, X., Wan, S., and Guizani, M. (2023). A two-stage strategy for UAV-enabled wireless power transfer in unknown environments. IEEE Trans. Mob. Comput., in press.10.1109/TMC.2023.3240763 ↗
13. Abbas, N., Abbas, Z., Liu, X., Khan, S.S., Foster, E.D., and Larkin, S. (2023). A Survey: Future Smart Cities Based on Advance Control of Unmanned Aerial Vehicles (UAVs). Appl. Sci., 13.10.3390/app13179881 ↗
14. Low-altitude unmanned aerial vehicles-based internet of things services: Comprehensive survey and future perspectives10.1109/JIOT.2016.2612119 ↗
15. A mobility compensation method for drones in SG-eIoT10.1016/j.dcan.2020.07.011 ↗
16. Markets (2022). UAV Market by Point of Sale, Systems, Platform (Civil & Commercial, and Defense & Government), Function, End Use, Application, Type (Fixed Wing, Rotary Wing, Hybrid), Mode of Operation, Mtow, Range & Region-Global Forecast to 2027, Research and Markets. Technical Report.
17. A Comprehensive Review of Unmanned Aerial Vehicle Attacks and Neutralization Techniques10.1016/j.adhoc.2020.102324 ↗
18. Li, S., Chai, Y., Guo, M., and Liu, Y. (2020, January 27–29). Research on Detection Method of UAV Based on micro-Doppler Effect. Proceedings of the 2020 39th Chinese Control Conference (CCC), Shenyang, China.10.23919/CCC50068.2020.9189414 ↗
19. Pappu, C.S., Beal, A.N., and Flores, B.C. (2021). Chaos based frequency modulation for joint monostatic and bistatic radar-communication systems. Remote Sens., 13.10.3390/rs13204113 ↗
20. Abd, M.H., Al-Suhail, G.A., Tahir, F.R., Ali Ali, A.M., Abbood, H.A., Dashtipour, K., Jamal, S.S., and Ahmad, J. (2022). Synchronization of monostatic radar using a time-delayed chaos-based FM waveform. Remote Sens., 14.10.3390/rs14091984 ↗
21. Mehta, S., Rastegari, M., Caspi, A., Shapiro, L., and Hajishirzi, H. (2018, January 8–14). Espnet: Efficient spatial pyramid of dilated convolutions for semantic segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01249-6_34 ↗
22. Anti-Drone System with Multiple Surveillance Technologies: Architecture, Implementation, and Challenges10.1109/MCOM.2018.1700430 ↗
23. Farlik, J., Kratky, M., Casar, J., and Stary, V. (2016, January 7–9). Radar cross section and detection of small unmanned aerial vehicles. Proceedings of the 2016 17th International Conference on Mechatronics-Mechatronika (ME), Prague, Czech Republic.
24. Spectrum and energy efficient multi-antenna spectrum sensing for green UAV communication10.1016/j.dcan.2022.09.017 ↗
25. Deep learning-based strategies for the detection and tracking of drones using several cameras10.1186/s41074-019-0059-x ↗
26. Infrared small UAV target detection based on residual image prediction via global and local dilated residual networks
27. Al-Emadi, S., Al-Ali, A., Mohammad, A., and Al-Ali, A. (2019, January 24–28). Audio based drone detection and identification using deep learning. Proceedings of the 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC), Tangier, Morocco.10.1109/IWCMC.2019.8766732 ↗
28. Development of an acoustic system for UAV detection10.3390/s20174870 ↗
29. UAV detection and identification based on WiFi signal and RF fingerprint10.1109/JSEN.2021.3068444 ↗
30. Detection of non-Gaussian signals using integrated polyspectrum10.1109/78.330373 ↗
31. Yao, Y., Yu, L., and Chen, Y. (2020, January 28–31). Specific Emitter Identification Based on Square Integral Bispectrum Features. Proceedings of the 2020 IEEE 20th International Conference on Communication Technology (ICCT), Nanning, China.10.1109/ICCT50939.2020.9295681 ↗
32. UAV detection and localization based on multi-dimensional signal features10.1109/JSEN.2021.3105229 ↗
33. Mo, Y., Huang, J., and Qian, G. (2022, January 9–12). UAV Tracking by Identification Using Deep Convolutional Neural Network. Proceedings of the 2022 IEEE 8th International Conference on Computer and Communications (ICCC), Chengdu, China.10.1109/ICCC56324.2022.10065721 ↗
34. Swinney, C.J., and Woods, J.C. (2022). Low-Cost Raspberry-Pi-Based UAS Detection and Classification System Using Machine Learning. Aerospace, 9.10.3390/aerospace9120738 ↗
35. Lu, S., Wang, W., Zhang, M., Li, B., Han, Y., and Sun, D. (2022, January 24–26). Detect the Video Recording Act of UAV through Spectrum Recognition. Proceedings of the 2022 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), Dalian, China.10.1109/ICAICA54878.2022.9844471 ↗
36. He, Z., Huang, J., and Qian, G. (2022, January 9–12). UAV Detection and Identification Based on Radio Frequency Using Transfer Learning. Proceedings of the 2022 IEEE 8th International Conference on Computer and Communications (ICCC), Virtual.10.1109/ICCC56324.2022.10065628 ↗
37. Li, T., Hong, Z., Cai, Q., Yu, L., Wen, Z., and Yang, R. (2021). Bissiam: Bispectrum siamese network based contrastive learning for uav anomaly detection. IEEE Trans. Knowl. Data Eng.10.1109/TKDE.2021.3118727 ↗
38. Drone sound detection system based on feature result-level fusion using deep learning10.1007/s11042-022-12964-3 ↗
39. Development of a new Light-Weight Convolutional Neural Network for acoustic-based amateur drone detection10.1016/j.apacoust.2022.108773 ↗
40. Persistent homology and the upper box dimension10.1007/s00454-019-00145-3 ↗
41. Approach to an irregular time series on the basis of the fractal theory10.1016/0167-2789(88)90081-4 ↗
42. A new feature vector using selected bispectra for signal classification with application in radar target recognition10.1109/78.942617 ↗
43. RF-based drone detection and identification using deep learning approaches: An initiative towards a large open source drone database10.1016/j.future.2019.05.007 ↗
44. Deep Learning Approach to UAV Detection and Classification by Using Compressively Sensed RF Signal10.3390/s22083072 ↗
45. DroneRF dataset: A dataset of drones for RF-based detection, classification and identification10.1016/j.dib.2019.104313 ↗
46. Ezuma, M., Erden, F., Anjinappa, C.K., Ozdemir, O., and Guvenc, I. (2023, June 01). Drone Remote Controller RF Signal Dataset. Available online: https://ieee-dataport.org/open-access/drone-remote-controller-rf-signal-dataset.
47. Vuorenmaa, M., Marin, J., Heino, M., Turunen, M., and Riihonen, T. (2023, June 01). Radio-Frequency Control and Video Signal Recordings of Drones. Available online: https://zenodo.org/record/4264467.
48. Basak, S., Rajendran, S., Pollin, S., and Scheers, B. (2021, January 5–9). Drone classification from RF fingerprints using deep residual nets. Proceedings of the 2021 International Conference on COMmunication Systems & NETworkS (COMSNETS), Bangalore, India.10.1109/COMSNETS51098.2021.9352891 ↗
49. Swinney, C.J., and Woods, J.C. (2023, June 01). DroneDetect Dataset: A Radio Frequency dataset of Unmanned Aerial System (UAS) Signals for Machine Learning Detection & Classification. Available online: https://ieee-dataport.org/open-access/dronedetect-dataset-radio-frequency-dataset-unmanned-aerial-system-uas-signals-machine.
50. Single and multiple drones detection and identification using RF based deep learning algorithm10.1016/j.eswa.2021.115928 ↗
51. Medaiyese, O., Ezuma, M., Lauf, A., and Adeniran, A. (2023, June 01). Cardinal RF (CardRF): An Outdoor UAV/UAS/Drone RF Signals with Bluetooth and WiFi Signals Dataset. Available online: https://ieee-dataport.org/documents/cardinal-rf-cardrf-outdoor-uavuasdrone-rf-signals-bluetooth-and-wifi-signals-dataset.
52. Drone Detection and Tracking in Real-Time by Fusion of Different Sensing Modalities10.3390/drones6110317 ↗
53. Detection of Multiple Drones in a Time-Varying Scenario Using Acoustic Signals10.3390/su14074041 ↗
54. Malicious UAV detection using integrated audio and visual features for public safety applications10.3390/s20143923 ↗
55. Classification, positioning, and tracking of drones by HMM using acoustic circular microphone array beamforming10.1186/s13638-019-1632-9 ↗
56. Gupta, H., and Gupta, D. (2016, January 14–15). LPC and LPCC method of feature extraction in Speech Recognition System. Proceedings of the 2016 6th International Conference-Cloud System and Big Data Engineering (Confluence), Noida, India.10.1109/CONFLUENCE.2016.7508171 ↗
57. Amateur Drones Detection: A machine learning approach utilizing the acoustic signals in the presence of strong interference10.1016/j.comcom.2020.02.065 ↗
58. Al-Emadi, S.A., Al-Ali, A.K., Al-Ali, A., and Mohamed, A. (2019, January 24–28). Audio Based Drone Detection and Identification using Deep Learning. Proceedings of the IWCMC 2019 Vehicular Symposium (IWCMC-VehicularCom 2019), Tangier, Morocco.10.1109/IWCMC.2019.8766732 ↗
59. Acoustic-based UAV detection using late fusion of deep neural networks10.3390/drones5030054 ↗
60. Robust real-time face detection10.1023/B:VISI.0000013087.49260.fb ↗
61. Dalal, N., and Triggs, B. (2005, January 20–26). Histograms of oriented gradients for human detection. Proceedings of the CVPR’05, San Diego, CA, USA.
62. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the CVPR, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
63. Spatial pyramid pooling in deep convolutional networks for visual recognition10.1109/TPAMI.2015.2389824 ↗
64. Girshick, R. (2015, January 13–16). Fast r-cnn. Proceedings of the ICCV, Santiago, Chile.10.1109/ICCV.2015.169 ↗
65. Faster r-CNN: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
66. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the CVPR, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
67. DRONET: Multi-Tasking Framework for Real-Time Industrial Facility Aerial Surveillance and Safety10.3390/drones6020046 ↗
68. Wang, J., Hongjun, W., Liu, J., Zhou, R., Chen, C., and Liu, C. (2022, January 1–3). Fast and Accurate Detection of UAV Objects Based on Mobile-Yolo Network. Proceedings of the 2022 14th International Conference on Wireless Communications and Signal Processing (WCSP), Virtually.10.1109/WCSP55476.2022.10039216 ↗
69. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the ECCV, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
70. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the ICCV, Venice, Italy.10.1109/ICCV.2017.324 ↗
71. The pascal visual object classes (voc) challenge10.1007/s11263-009-0275-4 ↗
72. The pascal visual object classes challenge: A retrospective10.1007/s11263-014-0733-5 ↗
73. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the ECCV, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
74. Real world object detection dataset for quadcopter unmanned aerial vehicle detection10.1109/ACCESS.2020.3026192 ↗
75. Air-to-air visual detection of micro-uavs: An experimental evaluation of deep learning10.1109/LRA.2021.3056059 ↗
76. Walter, V., Vrba, M., and Saska, M. (August, January 31). On training datasets for machine learning-based visual relative localization of micro-scale UAVs. Proceedings of the ICRA, Paris, France.
77. Chen, Y., Aggarwal, P., Choi, J., and Kuo, C.C.J. (2017, January 12–15). A deep learning approach to drone monitoring. Proceedings of the APSIPA ASC, Kuala Lumpur, Malaysia.10.1109/APSIPA.2017.8282120 ↗
78. Vision-Based Anti-UAV Detection and Tracking10.1109/TITS.2022.3177627 ↗
79. Jiang, N., Wang, K., Peng, X., Yu, X., Wang, Q., Xing, J., Li, G., Zhao, J., Guo, G., and Han, Z. (2021). Anti-UAV: A large multi-modal benchmark for UAV tracking. arXiv.
80. Bosquet, B., Mucientes, M., and Brea, V. (2018, January 3–6). STDnet: A ConvNet for Small Target Detection. Proceedings of the 29th British Machine Vision Conference, Newcastle, UK.
81. Ajakwe, S.O., Saviour, I.I., Kim, J.H., Kim, D.S., and Lee, J.M. (2023, January 4–7). BANDA: A Novel Blockchain-Assisted Network for Drone Authentication. Proceedings of the 2023 Fourteenth International Conference on Ubiquitous and Future Networks (ICUFN), Paris, France.10.1109/ICUFN57995.2023.10201012 ↗
82. A Secure Intrusion Detection Platform Using Blockchain and Radial Basis Function Neural Networks for Internet of Drones10.1109/JIOT.2023.3237661 ↗
83. RF-UAVNet: High-performance convolutional network for RF-based drone surveillance systems10.1109/ACCESS.2022.3172787 ↗
84. TIB-Net: Drone detection network with tiny iterative backbone10.1109/ACCESS.2020.3009518 ↗
85. Liu, Y., Liu, D., Wang, B., and Chen, B. (2022, January 22–24). Mob-YOLO: A Lightweight UAV Object Detection Method. Proceedings of the 2022 International Conference on Sensing, Measurement & Data Analytics in the Era of Artificial Intelligence (ICSMD), Harbin, China.10.1109/ICSMD57530.2022.10058230 ↗
86. Golam, M., Akter, R., Naufal, R., Doan, V.S., Lee, J.M., and Kim, D.S. (December, January 28). Blockchain Inspired Intruder UAV Localization Using Lightweight CNN for Internet of Battlefield Things. Proceedings of the MILCOM 2022—2022 IEEE Military Communications Conference (MILCOM), Norfolk, VA, USA.10.1109/MILCOM55135.2022.10017795 ↗
87. DoA estimation using neural network-based covariance matrix reconstruction10.1109/LSP.2021.3072564 ↗
88. A RBFNN approach for DoA estimation of ultra wideband antenna array10.1016/j.neucom.2007.08.023 ↗
89. Deep Convolution Network for Direction of Arrival Estimation With Sparse Prior10.1109/LSP.2019.2945115 ↗
90. RFDOA-Net: An Efficient ConvNet for RF-Based DOA Estimation in UAV Surveillance Systems10.1109/TVT.2021.3114058 ↗
91. Beamforming: A versatile approach to spatial filtering10.1109/53.665 ↗
92. Multiple emitter location and signal parameter estimation10.1109/TAP.1986.1143830 ↗
93. ESPRIT-estimation of signal parameters via rotational invariance techniques10.1109/29.32276 ↗
94. Coherent SVR learning for wideband direction-of-arrival estimation10.1109/LSP.2019.2901641 ↗
95. Indoor sound source localization with probabilistic neural network10.1109/TIE.2017.2786219 ↗
96. Multi-speaker DOA estimation using deep convolutional networks trained with noise signals10.1109/JSTSP.2019.2901664 ↗
97. Acoustic localization and tracking of a multi-rotor unmanned aerial vehicle using an array with few microphones10.1121/10.0001930 ↗
98. A geometric approach to sound source localization from time-delay estimates10.1109/TASLP.2014.2317989 ↗
99. Chen, J., Zhao, Y., Zhao, C., and Zhao, Y. (2018, January 20–22). Improved two-step weighted least squares algorithm for TDOA-based source localization. Proceedings of the 2018 19th International Radar Symposium (IRS), Piscataway, NJ, USA.10.23919/IRS.2018.8448149 ↗
100. An acoustic-based surveillance system for amateur drones detection and localization10.1109/TVT.2020.2964110 ↗