Local Attention Sequence Model for Video Object Detection
Source abstract
Abstract
Video object detection still faces several difficulties and challenges. For example, the imbalance of positive and negative samples leads to low information processing efficiency, and detection performance declines in abnormal situations in video. This paper examines video object detection based on local attention to address such challenges. We propose a local attention sequence model and optimized the parameter and calculation of ConvGRU. It could process spatial and temporal information in videos more efficiently and ultimately improve detection performance under abnormal conditions. The experiments on ImageNet VID show that our method could improve the detection accuracy by 5.3%, and the visualization results show that the method is adaptive to different abnormal conditions, thereby improving the reliability of video object detection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Visual Attention and Saliency Detection · Advanced Neural Network Applications · Advanced Image and Video Retrieval Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China
Technology and Equipment of Rail Transit Operation and Maintenance Key Laboratory of Sichuan Province, Chengdu 610031, China
School of Mechanical Engineering, Southwest Jiaotong University, Chengdu 610031, China
The State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou 310027, China
Funding
Open Research Project of Technology and Equipment of Rail Transit Operation and Maintenance Key Laboratory of Sichuan Province
References (25)
Show source-provided references
1. Deep learning for generic object detection: A survey10.1007/s11263-019-01247-4 ↗
2. Zou, Z., Shi, Z., Guo, Y., and Ye, J. (2019). Object Detection in 20 Years: A Survey. arXiv.
3. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
4. Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Adv. Neural Inf. Process. Syst., 1137–1149.10.1109/TPAMI.2016.2577031 ↗
5. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
6. Dosovitskiy, A., Fischer, P., Ilg, E., Häusser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D., and Brox, T. (2015, January 7–13). Flownet: Learning Optical Flow with Convolutional Networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.316 ↗
7. Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., and Brox, T. (2017, January 21–26). Flownet 2.0: Evolution of Optical Flow Estimation with Deep Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.179 ↗
8. Huang, B., Huang, H., and Lu, H. (2017). Convolutional Gated Recurrent Units Fusion for Video Action Recognition. International Conference on Neural Information Processing, Springer International Publishing.10.1007/978-3-319-70090-8_12 ↗
9. Lin, T., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature Pyramid Networks for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
10. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single Shot MultiBox Detector. European Conference on Computer Vision, Springer.10.1007/978-3-319-46448-0_2 ↗
11. Lin, T., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal Loss for Dense Object Detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
12. T-CNN: Tubelets with Convolutional Neural Networks for Object Detection From Videos10.1109/TCSVT.2017.2736553 ↗
13. Han, W., Khorrami, P., Paine, T.L., Ramachandran, P., Babaeizadeh, M., Shi, H., Li, J., Yan, S., and Huang, T.S. (2016). Seq-NMS for Video Object Detection. arXiv.
14. Zhu, X., Wang, Y., Dai, J., Yuan, L., and Wei, Y. (2017, January 22–29). Flow-Guided Feature Aggregation for Video Object Detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.52 ↗
15. Zhu, X., Dai, J., Yuan, L., and Wei, Y. (2018, January 18–23). Towards High Performance Video Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00753 ↗
16. Liu, M., and Zhu, M. (2018, January 18–23). Mobile Video Object Detection with Temporally-Aware Feature Maps. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.
17. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
18. Xiao, F., and Lee, Y.J. (2018, January 8–14). Video Object Detection with an Aligned Spatial-Temporal Memory. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01237-3_30 ↗
19. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is All You Need. Advances in Neural Information Processing Systems, The MIT Press.
20. Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2015). Spatial Transformer Networks. arXiv.
21. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-Excitation Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
22. Wang, X., Girshick, R., Gupta, A., and He, K. (2018, January 18–23). Non-Local Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00813 ↗
23. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
24. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18–23). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
25. Imagenet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗