Object Detection in Drone Imagery via Sample Balance Strategies and Local Feature Enhancement
Source abstract
Abstract
With the advent of drones, new potential applications have emerged for the unconstrained analysis of images and videos from aerial view cameras. Despite the tremendous success of the generic object detection methods developed using ground-based photos, a considerable performance drop is observed when these same methods are directly applied to images captured by Unmanned Aerial Vehicles (UAVs). Usually, most of the work goes into improving the performance of the detector in aspects such as design loss, training sample selection, feature enhancement, and so forth. This paper proposes a detection framework based on an anchor-free detector with several modules, including a sample balance strategies module and super-resolved generated feature module, to improve performance. We proposed the sample balance strategies module to optimize the imbalance among training samples, especially the imbalance between positive and negative, and easy and hard samples. Due to the high frequencies and noisy representation of the small objects in images captured by drones, the detection task is extraordinarily challenging. However, when compared with other algorithms of this kind, our method achieves better results. We also propose a super-resolved generated GAN (Generative Adversarial Network) module with center-ness weights to effectively enhance the local feature map. Finally, we demonstrate our method’s effectiveness with the proposed modules by carrying out a state-of-the-art performance on Visdrone2020 benchmarks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Image and Video Retrieval Techniques · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, China
University of Chinese Academy of Sciences, Beijing 100049, China
References (56)
Show source-provided references
1. Advanced deep-learning techniques for salient and category-specific object detection: A survey10.1109/MSP.2017.2749125 ↗
2. Ground-based image analysis: A tutorial on machine-learning techniques and applications10.1109/MGRS.2015.2510448 ↗
3. Leal-Taixé, L., and Roth, S. (2019). Computer Vision–ECCV 2018 Workshops: Munich, Germany, September 8–14, 2018, Proceedings, Part VI, Springer.10.1007/978-3-030-11024-6 ↗
4. Angelova, A., and Zhu, S. (2013, January 23–28). Efficient object detection and segmentation for fine-grained recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.110 ↗
5. Voigtlaender, P., Krause, M., Osep, A., Luiten, J., Sekar, B.B.G., Geiger, A., and Leibe, B. (2019, January 15–20). Mots: Multi-object tracking and segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00813 ↗
6. A survey of recent advances in cnn-based single image crowd counting and density estimation10.1016/j.patrec.2017.07.007 ↗
7. Ma, Y., Wu, X., Yu, G., Xu, Y., and Wang, Y. (2016). Pedestrian detection and tracking from low-resolution unmanned aerial vehicle thermal imagery. Sensors, 16.10.3390/s16040446 ↗
8. Small object detection in unmanned aerial vehicle images using feature fusion and scaling-based single shot detector with spatial context analysis10.1109/TCSVT.2019.2905881 ↗
9. Wu, Z., Suresh, K., Narayanan, P., Xu, H., Kwon, H., and Wang, Z. (2019, January 27–28). Delving into robust object detection from unmanned aerial vehicles: A deep nuisance disentanglement approach. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.10.1109/ICCV.2019.00129 ↗
10. Du, D., Qi, Y., Yu, H., Yang, Y., Duan, K., Li, G., Zhang, W., Huang, Q., and Tian, Q. (2018, January 8–14). The unmanned aerial vehicle benchmark: Object detection and tracking. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01249-6_23 ↗
11. Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., and Ren, D. (2020, January 7–12). Distance-IoU loss: Faster and better learning for bounding box regression. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i07.6999 ↗
12. Zhu, P., Wen, L., Du, D., Bian, X., Hu, Q., and Ling, H. (2020). Vision meets drones: Past, present and future. arXiv.
13. Convolutional SVM networks for object detection in UAV imagery10.1109/TGRS.2018.2790926 ↗
14. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks10.1109/TPAMI.2016.2577031 ↗
15. G-CNN: Object detection via grid convolutional neural network10.1109/ACCESS.2017.2770178 ↗
16. Object detection based on multi-layer convolution feature fusion and online hard example mining10.1109/ACCESS.2018.2815149 ↗
17. Dai, J., Li, Y., He, K., and Sun, J. (2018). R-FCN: Object detection via region-based fully convolutional networks [EB/OL]. arXiv.
18. Li, Y., Chen, Y., Wang, N., and Zhang, Z. (2019, January 27–28). Scale-aware trident networks for object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.10.1109/ICCV.2019.00615 ↗
19. Kong, T., Yao, A., Chen, Y., and Sun, F. (2016, January 27–30). HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.98 ↗
20. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
21. Tan, Z., Nie, X., Qian, Q., Li, N., and Li, H. (2019, January 27–28). Learning to Rank Proposals for Object Detection. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.10.1109/ICCV.2019.00836 ↗
22. Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W., and Lin, D. (2019, January 15–20). Libra R-CNN: Towards Balanced Learning for Object Detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00091 ↗
23. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 8–16). SSD: Single Shot MultiBox Detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
24. Chen, K., Li, J., Lin, W., See, J., and Zou, J. (2019, January 15–20). Towards Accurate One-Stage Object Detection with AP-Loss. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00526 ↗
25. Li, S., Yang, L., Huang, J., Hua, X.S., and Zhang, L. (2019, January 27–28). Dynamic Anchor Feature Selection for Single-Shot Object Detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.10.1109/ICCV.2019.00671 ↗
26. Liu, S., and Huang, D. (2018, January 8–14). Receptive field block net for accurate and fast object detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01252-6_24 ↗
27. Law, H., and Deng, J. (2018, January 8–14). Cornernet: Detecting objects as paired keypoints. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_45 ↗
28. Law, H., Teng, Y., Russakovsky, O., and Deng, J. (2019). Cornernet-lite: Efficient keypoint based object detection. arXiv.
29. Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (2019, January 27–28). Centernet: Keypoint triplets for object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.10.1109/ICCV.2019.00667 ↗
30. Zhou, X., Zhuo, J., and Krahenbuhl, P. (2019, January 15–20). Bottom-up object detection by grouping extreme and center points. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00094 ↗
31. Zhou, X., Wang, D., and Krähenbühl, P. (2019). Objects as points. arXiv.
32. Yang, Z., Liu, S., Hu, H., Wang, L., and Lin, S. (2019, January 27–28). Reppoints: Point set representation for object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.10.1109/ICCV.2019.00975 ↗
33. Shafiee, M.J., Chywl, B., Li, F., and Wong, A. (2017). Fast YOLO: A fast you only look once system for real-time embedded object detection in video. arXiv.10.15353/vsnl.v3i1.171 ↗
34. Huang, L., Yang, Y., Deng, Y., and Yu, Y. (2015). Densebox: Unifying landmark localization with end to end object detection. arXiv.
35. Zhu, C., He, Y., and Savvides, M. (2019, January 15–20). Feature selective anchor-free module for single-shot object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00093 ↗
36. Foveabox: Beyound anchor-based object detection10.1109/TIP.2020.3002345 ↗
37. Tian, Z., Shen, C., Chen, H., and He, T. (2019, January 27–28). Fcos: Fully convolutional one-stage object detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.10.1109/ICCV.2019.00972 ↗
38. Shrivastava, A., Gupta, A., and Girshick, R. (2016, January 27–30). Training region-based object detectors with online hard example mining. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.89 ↗
39. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
40. Zhang, S., Chi, C., Yao, Y., Lei, Z., and Li, S.Z. (2020, January 13–19). Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00978 ↗
41. Generative Adversarial Networks
42. Denton, E., Chintala, S., Szlam, A., and Fergus, R. (2015). Deep generative image models using a laplacian pyramid of adversarial networks. arXiv.
43. Li, C., and Wand, M. (2016, January 27–30). Combining markov random fields and convolutional neural networks for image synthesis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.272 ↗
44. Yeh, R., Chen, C., Lim, T., Hasegawa-Johnson, M., and Do, M. (2016). Semantic image inpainting with perceptual and contextual losses, vol. 2. arXiv.10.1109/CVPR.2017.728 ↗
45. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21–26). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.19 ↗
46. Li, J., Liang, X., Wei, Y., Xu, T., Feng, J., and Yan, S. (2017, January 21–26). Perceptual generative adversarial networks for small object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.211 ↗
47. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
48. The pascal visual object classes challenge: A retrospective10.1007/s11263-014-0733-5 ↗
49. Imagenet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗
50. Mundhenk, T.N., Konjevod, G., Sakla, W.A., and Boakye, K. (2016, January 8–16). A large contextual dataset for classification, detection and counting of cars with deep learning. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46487-9_48 ↗
51. Hsieh, M.R., Lin, Y.L., and Hsu, W.H. (2017, January 22–29). Drone-based object counting by spatially regularized regional proposal network. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.446 ↗
52. Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., and Zhang, L. (2018, January 18–22). DOTA: A large-scale dataset for object detection in aerial images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00418 ↗
53. Targ, S., Almeida, D., and Lyman, K. (2016). Resnet in resnet: Generalizing residual architectures. arXiv.
54. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). Pytorch: An imperative style, high-performance deep learning library. arXiv.
55. Chen, K., Wang, J., Pang, J., Cao, Y., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., and Xu, J. (2019). MMDetection: Open mmlab detection toolbox and benchmark. arXiv.
56. Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Loy, C.C., Qiao, Y., and Tang, X. (2018, January 8–14). ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany.10.1007/978-3-030-11021-5_5 ↗