SSD7-FFAM: A Real-Time Object Detection Network Friendly to Embedded Devices from Scratch
Source abstract
Abstract
The high requirements for computing and memory are the biggest challenges in deploying existing object detection networks to embedded devices. Living lightweight object detectors directly use lightweight neural network architectures such as MobileNet or ShuffleNet pre-trained on large-scale classification datasets, which results in poor network structure flexibility and is not suitable for some specific scenarios. In this paper, we propose a lightweight object detection network Single-Shot MultiBox Detector (SSD)7-Feature Fusion and Attention Mechanism (FFAM), which saves storage space and reduces the amount of calculation by reducing the number of convolutional layers. We offer a novel Feature Fusion and Attention Mechanism (FFAM) method to improve detection accuracy. Firstly, the FFAM method fuses high-level semantic information-rich feature maps with low-level feature maps to improve small objects’ detection accuracy. The lightweight attention mechanism cascaded by channels and spatial attention modules is employed to enhance the target’s contextual information and guide the network to focus on its easy-to-recognize features. The SSD7-FFAM achieves 83.7% mean Average Precision (mAP), 1.66 MB parameters, and 0.033 s average running time on the NWPU VHR-10 dataset. The results indicate that the proposed SSD7-FFAM is more suitable for deployment to embedded devices for real-time object detection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · COVID-19 diagnosis using AI · Advanced Image and Video Retrieval Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Chongqing Key Laboratory of Space Information Network and Intelligent Information Fusion, School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400030, China
Funding
National Key Research and Development Program of China
References (51)
Show source-provided references
1. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA. (In English).10.1109/CVPR.2014.81 ↗
2. He, K.M., Gkioxari, G., Dollar, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy. (In English).10.1109/ICCV.2017.322 ↗
3. T-CNN: Tubelets with Convolutional Neural Networks for Object Detection From Videos10.1109/TCSVT.2017.2736553 ↗
4. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). SSD: Single Shot MultiBox Detector. Proceedings of the European Conference on Computer Vision—Eccv 2016, Pt I, Amsterdam, The Netherlands. (In English).10.1007/978-3-319-46448-0_2 ↗
5. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (Cvpr), Las Vegas, NV, USA. (In English).10.1109/CVPR.2016.91 ↗
6. Ren, S.Q., He, K.M., Girshick, R., and Sun, J. (2015, January 7–12). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Proceedings of the Advances in Neural Information Processing Systems 28 (NIPS 2015), Montreal, QC, Canada. (In English).
7. Lin, T.Y., Dollar, P., Girshick, R., He, K.M., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature Pyramid Networks for Object Detection. Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (Cvpr 2017), Honolulu, HI, USA. (In English).10.1109/CVPR.2017.106 ↗
8. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, Faster, Stronger. Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (Cvpr 2017), Honolulu, HI, USA. (In English).10.1109/CVPR.2017.690 ↗
9. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental improvement. arXiv, (In English).
10. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017, January 21–26). MobileNets: Efficient Convolutional Neural Networks for mobile vision applications. Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (Cvpr), Honolulu, HI, USA. (In English).
11. Sandler, M., Howard, A., Zhu, M.L., Zhmoginov, A., and Chen, L.C. (2018, January 18–22). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (Cvpr), Salt Lake City, UT, USA. (In English).10.1109/CVPR.2018.00474 ↗
12. Zhang, X., Zhou, X.Y., Lin, M.X., and Sun, R. (2018, January 18–22). ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (Cvpr), Salt Lake City, UT, USA. (In English).10.1109/CVPR.2018.00716 ↗
13. Forrest, N.I., Song, H., Matthew, W.M., Khalid, A., and William, J.D. (2017, January 24–26). SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and <0.5 MB model size. Proceedings of the 5th International Conference on Learning Representations, Toulon, France.
14. Yin, R., Zhao, W., Fan, X., and Yin, Y. (2020). AF-SSD: An Accurate and Fast Single Shot Detector for High Spatial Remote Sensing Imagery. Sensors, 20.10.3390/s20226530 ↗
15. Ma, N., Zhang, X., Zheng, H.-T., and Sun, J. (2018, January 8–14). Shufflenet V2: Practical guidelines for efficient cnn architecture design. Proceedings of the 2018 European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01264-9_8 ↗
16. Womg, A., Shafiee, M.J., Li, F., and Chwyl, B. (2018, January 8–10). Tiny SSD: A Tiny Single-Shot Detection Deep Convolutional Neural Network for Real-Time Embedded Object Detection. Proceedings of the 2018 15th Conference on Computer and Robot Vision (CRV), Toronto, ON, Canada.10.1109/CRV.2018.00023 ↗
17. Wang, R.J., Li, X., and Ling, C.X. (2018, January 3–8). Pelee: A Real-Time Object Detection System on Mobile Devices. Proceedings of the Advances in Neural Information Processing Systems 31 (Nips 2018), Montréal, QC, Canada. (In English).
18. Singh, B., and Davis, L.S. (2018, January 18–22). An Analysis of Scale Invariance in Object Detection—SNIP. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (Cvpr), Salt Lake City, UT, USA. (In English).10.1109/CVPR.2018.00377 ↗
19. Peng, C., Xiao, T., Li, Z., Jiang, Y., Zhang, X., Jia, K., Yu, G., and Sun, J. (2018, January 18–22). MegDet: A Large Mini-Batch Object Detector. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (Cvpr), Salt Lake City, UT, USA. (In English).10.1109/CVPR.2018.00647 ↗
20. Kong, T., Yao, A.B., Chen, Y.R., and Sun, F.C. (2016, January 27–30). HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (Cvpr), Las Vegas, NV, USA. (In English).10.1109/CVPR.2016.98 ↗
21. Kong, T., Sun, F.C., Yao, A.B., Liu, H.P., Lu, M., and Chen, Y.R. (2017, January 21–26). RON: Reverse Connection with Objectness Prior Networks for Object Detection. Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (Cvpr 2017), Honolulu, HI, USA. (In English).10.1109/CVPR.2017.557 ↗
22. Bosquet, B., Mucientes, M., and Brea, V.M. (2020). STDnet: Exploiting high resolution feature maps for small object detection. Eng. Appl. Artif. Intell., 91, (In English).10.1016/j.engappai.2020.103615 ↗
23. Bell, S., Zitnick, C.L., Bala, K., and Girshick, R. (2016, January 27–30). Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (Cvpr), Las Vegas, NV, USA. (In English).10.1109/CVPR.2016.314 ↗
24. Wu, B.C., Iandola, F., Jin, P.H., and Keutzer, K. (2017, January 21–26). SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA. (In English).10.1109/CVPRW.2017.60 ↗
25. Zhang, S., Wen, L.Y., Bian, X., Lei, Z., and Li, S.Z. (2018, January 18–22). Single-Shot Refinement Neural Network for Object Detection. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (Cvpr), Salt Lake City, UT, USA. (In English).10.1109/CVPR.2018.00442 ↗
26. Lin, T.Y., Goyal, P., Girshick, R., He, K.M., and Dollar, P. (2017, January 22–29). Focal Loss for Dense Object Detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy. (In English).10.1109/ICCV.2017.324 ↗
27. CornerNet: Detecting Objects as Paired Keypoints10.1007/s11263-019-01204-1 ↗
28. Girshick, R. (2015, January 7–13). Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile. (In English).10.1109/ICCV.2015.169 ↗
29. Cao, G.M., Xie, X.M., Yang, W.Z., Liao, Q., Shi, G.M., and Wu, J.J. (2017, January 13–15). Feature-Fused SSD: Fast Detection for Small Objects. Proceedings of the Ninth International Conference on Graphic and Image Processing (ICGIP 2017), Qingdao, China. (In English).
30. Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2015, January 7–12). Spatial Transformer Networks. Proceedings of the Advances in Neural Information Processing Systems 28 (NIPS 2015), Montreal, QC, Canada. (In English).
31. Hu, J., Shen, L., and Sun, G. (2018, January 18–22). Squeeze-and-Excitation Networks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (Cvpr), Salt Lake City, UT, USA. (In English).10.1109/CVPR.2018.00745 ↗
32. Diversified Visual Attention Networks for Fine-Grained Object Classification10.1109/TMM.2017.2648498 ↗
33. Pierluigiferrarr (2018, May 03). keras_ssd7. Available online: https://github.com/pierluigiferrari/ssd_keras/blob/master/models/keras_ssd7.py.
34. Squeeze-and-Excitation Networks10.1109/TPAMI.2019.2913372 ↗
35. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2016, January 27–30). Learning Deep Features for Discriminative Localization. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (Cvpr), Las Vegas, NV, USA. (In English).10.1109/CVPR.2016.319 ↗
36. Zagoruyko, S., and Komodakis, N. (2017, January 24–26). Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. Proceedings of the ICLR 2017, Toulon, France.
37. Multi-class geospatial object detection and geographic image classification based on collection of part detectors10.1016/j.isprsjprs.2014.10.002 ↗
38. The Pascal Visual Object Classes (VOC) Challenge10.1007/s11263-009-0275-4 ↗
39. ADN for object detection10.1049/iet-cvi.2018.5651 ↗
40. Glorot, X., and Bengio, Y. (2010, January 13–15). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS), Sardinia, Italy.
41. Depth Estimation by Parameter Transfer with a Lightweight Model for Single Still Images10.1109/TCSVT.2016.2580438 ↗
42. Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images10.1109/TGRS.2016.2601622 ↗
43. Han, X., Zhong, Y., and Zhang, L. (2017). An Efficient and Robust Integrated Geospatial Object Detection Framework for High Spatial Resolution Remote Sensing Imagery. Remote Sens., 9.10.3390/rs9070666 ↗
44. Xie, W., Qin, H., Li, Y., Wang, Z., and Lei, J. (2019). A Novel Effectively Optimized One-Stage Network for Object Detection in Remote Sensing Imagery. Remote Sens., 11.10.3390/rs11111376 ↗
45. Yamashige, Y., and Aono, M. (2019, January 20–21). FPSSD7: Real-time Object Detection using 7 Layers of Convolution based on SSD. Proceedings of the 2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA), Yogyakarta, Indonesia.10.1109/ICAICTA.2019.8904089 ↗
46. Hwang, Y.J., Lee, J.G., Moon, U.C., and Park, H.H. (2020). SSD-TSEFFM: New SSD Using Trident Feature and Squeeze and Extraction Feature Fusion. Sensors, 20.10.3390/s20133630 ↗
47. Ryu, J., and Kim, S. (2019). Chinese Character Boxes: Single Shot Detector Network for Chinese Character Detection. Appl. Sci., 9, (In English).10.3390/app9020315 ↗
48. Gidaris, S., and Komodakis, N. (2015, January 11–18). Object detection via a multi-region & semantic segmentation-aware CNN model. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile. (In English).10.1109/ICCV.2015.135 ↗
49. Mehra, M., Sahai, V., Chowdhury, P., and Dsouza, E. (2019, January 20–21). Home Security System using IOT and AWS Cloud Services. Proceedings of the 2019 International Conference on Advances in Computing, Communication and Control (ICAC3), Mumbai, India.10.1109/ICAC347590.2019.9089839 ↗
50. Guillermo, M., Billones, R.K., Bandala, A., Vicerra, R.R., Sybingco, E., Dadios, E.P., and Fillone, A. (2020, January 16–19). Implementation of Automated Annotation through Mask RCNN Object Detection model in CVAT using AWS EC2 Instance. Proceedings of the 2020 IEEE Region 10 Conference (TENCON), Osaka, Japan.10.1109/TENCON50793.2020.9293906 ↗
51. Seal, A., and Mukherjee, A. (2019, January 11–14). Real Time Accident Prediction and Related Congestion Control Using Spark Streaming in an AWS EMR cluster. Proceedings of the 2019 SoutheastCon, Huntsville, AL, USA.10.1109/SoutheastCon42311.2019.9020661 ↗