Object Detection Based on Multiple Information Fusion Net
Source abstract
Abstract
Object detection has been playing a significant role in computer vision for a long time, but it is still full of challenges. In this paper, we propose a novel object detection framework based on relationship among different objects and the scene-level information of the whole image to cope with the problem that some strongly correlated objects are difficult to be recognized. Our motivation is to enrich the semantics of object detection feature by a scene-level information branch and a relationship branch. There are three important changes of our framework over traditional detection methods: representation of relationship, scene-level information as the prior knowledge and the fusion of the above two information. Extensive experiments are carried out on PASCAL VOC and MS COCO databases. The experimental results show that the detection performance can be improved by introducing relationship and scene-level information, and our proposed model achieve better performance than several classical and state-of-the-art methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Advanced Image and Video Retrieval Techniques · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Information Science and Technology, Northeast Normal University, Changchun 130000, China
Institute for Intelligent Elderlycare, College of Humanities and Sciences, Northeast Normal University, Changchun 130000, China
Funding
National Natural Science Foundation of China · Jilin Provincial Science and Technology Department Project · Fundamental Research Funds for the Central Universities
References (58)
Show source-provided references
1. Object detection with discriminatively trained part-based models10.1109/TPAMI.2009.167 ↗
2. Distinctive image features from scale-invariant keypoints10.1023/B:VISI.0000029664.99615.94 ↗
3. Dalal, N., and Triggs, B. (2005, January 20–25). Histograms of Oriented Gradients for Human Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA.
4. Bay, H., Tuytelaars, T., and Van Gool, L. (2006). Surf: Speeded up robust features. European Conference on Computer Vision, Springer.10.1007/11744023_32 ↗
5. Least squares support vector machine classifiers10.1023/A:1018628609742 ↗
6. A decision-theoretic generalization of on-line learning and an application to boosting10.1006/jcss.1997.1504 ↗
7. Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster r-cnn: Towards Real-Time Object Detection with Region Proposal Networks, Neural Information Processing Systems Foundation, Inc.
8. Selective search for object recognition10.1007/s11263-013-0620-5 ↗
9. Zitnick, C.L., and Dollár, P. (2014). Edge boxes: Locating object proposals from edges. European Conference on Computer Vision, Springer.10.1007/978-3-319-10602-1_26 ↗
10. Shi, W., Bao, S., and Tan, D. (2019). FFESSD: An Accurate and Efficient Single-Shot Detector for Target Detection. Appl. Sci., 9.10.3390/app9204276 ↗
11. He, W., Huang, Z., Wei, Z., Li, C., and Guo, B. (2019). TF-YOLO: An Improved Incremental Network for Real-Time Object Detection. Appl. Sci., 9.10.3390/app9163225 ↗
12. Pinheiro, P.O., Collobert, R., and Dollár, P. (2015, January 7–12). Learning to segment object candidates. Proceedings of the Conference on Neural Information Processing Systems, Montreal, QC, Canada.
13. Jiang, Y., Peng, T., and Tan, N. (2019). CP-SSD: Context Information Scene Perception Object Detection Based on SSD. Appl. Sci., 9.10.3390/app9142785 ↗
14. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
15. Spatial pyramid pooling in deep convolutional networks for visual recognition10.1109/TPAMI.2015.2389824 ↗
16. Girshick, R. (2015, January 7–13). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
17. Dai, J., Li, Y., He, K., and Sun, J. (2016, January 5–10). R-fcn: Object detection via region-based fully convolutional networks. Proceedings of the Conference on Neural Information Processing Systems, Barcelona, Spain.
18. Cai, Z., Fan, Q., Feris, R.S., and Vasconcelos, N. (2016). A unified multi-scale deep convolutional neural network for fast object detection. European Conference on Computer Vision, Springer.10.1007/978-3-319-46493-0_22 ↗
19. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
20. Ioffe, S., and Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv.
21. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗
22. Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2016). Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv.10.1609/aaai.v31i1.11231 ↗
23. Huang, J., and Rathod, V. (2020, January 04). Supercharge your Computer Vision Models with the TensorFlow Object Detection API. Google AI Blog, 15 June 2017. Available online: https://ai.googleblog.com/2017/06/supercharge-your-computervision-models.html.
24. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the International Conference on Computer Vision & Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
25. Redmon, J., and Farhadi, A. (2017). YOLO9000: Better, faster, stronger. arXiv.10.1109/CVPR.2017.690 ↗
26. Najibi, M., Rastegari, M., and Davis, L.S. (2016, January 27–30). G-cnn: An iterative grid based object detector. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.260 ↗
27. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). Ssd: Single shot multibox detector. European Conference on Computer Vision, Springer.10.1007/978-3-319-46448-0_2 ↗
28. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
29. Jae Hwang, S., Ravi, S.N., Tao, Z., Kim, H.J., Collins, M.D., and Singh, V. (2018, January 18–22). Tensorize, factorize and regularize: Robust visual relationship learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00112 ↗
30. Krishna, R., Chami, I., Bernstein, M., and Fei-Fei, L. (2018, January 18–22). Referring relationships. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00718 ↗
31. Chen, X., Li, L.J., Fei-Fei, L., and Gupta, A. (2018, January 18–22). Iterative visual reasoning beyond convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00756 ↗
32. Bell, S., Lawrence Zitnick, C., Bala, K., and Girshick, R. (2016, January 27–30). Inside-outside net: Detecting objects in context with skip pooling and recurrent neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.314 ↗
33. Xu, D., Zhu, Y., Choy, C.B., and Fei-Fei, L. (2017, January 21–26). Scene graph generation by iterative message passing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.330 ↗
34. Gkioxari, G., Girshick, R., Dollár, P., and He, K. (2018, January 18–22). Detecting and recognizing human-object interactions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00872 ↗
35. Liu, Y., Wang, R., Shan, S., and Chen, X. (2018, January 18–22). Structure inference net: Object detection using scene-level context and instance-level relationships. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00730 ↗
36. Hu, H., Gu, J., Zhang, Z., Dai, J., and Wei, Y. (2018, January 18–22). Relation networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00378 ↗
37. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
38. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the Conference and Workshop on Neural Information Processing Systems, Long Beach, CA, USA.
39. Zeng, X., Ouyang, W., Yang, B., Yan, J., and Wang, X. (2016). Gated bi-directional cnn for object detection. European Conference on Computer Vision, Springer.10.1007/978-3-319-46478-7_22 ↗
40. Shrivastava, A., and Gupta, A. (2016). Contextual priming and feedback for faster r-cnn. European Conference on Computer Vision, Springer.10.1007/978-3-319-46448-0_20 ↗
41. The pascal visual object classes (voc) challenge10.1007/s11263-009-0275-4 ↗
42. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014). Microsoft coco: Common objects in context. European Conference on Computer Vision, Springer.10.1007/978-3-319-10602-1_48 ↗
43. Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., and Pietikäinen, M. (2018). Deep learning for generic object detection: A survey. arXiv.10.1007/s11263-019-01247-4 ↗
44. Contextual priming for object detection10.1023/A:1023052124951 ↗
45. The role of context in object recognition10.1016/j.tics.2007.09.009 ↗
46. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2014). Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv.
47. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs10.1109/TPAMI.2017.2699184 ↗
48. Divvala, S.K., Hoiem, D., Hays, J.H., Efros, A.A., and Hebert, M. (2009, January 20–25). An empirical study of context in object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL, USA.10.1109/CVPRW.2009.5206532 ↗
49. Context based object categorization: A critical survey10.1016/j.cviu.2010.02.004 ↗
50. Object class detection: A survey10.1145/2522968.2522978 ↗
51. Ouyang, W., Wang, X., Zeng, X., Qiu, S., Luo, P., Tian, Y., Li, H., Yang, S., Wang, Z., and Loy, C.-C. (2015, January 7–12). Deepid-net: Deformable deep convolutional neural networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298854 ↗
52. Hu, D. (2019). An introductory survey on attention mechanisms in NLP problems. Proceedings of SAI Intelligent Systems Conference, Springer.10.1007/978-3-030-29513-4_31 ↗
53. Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y. (, January June). Show, attend and tell: Neural image caption generation with visual attention. Proceedings of the International Conference on Machine Learning, Ithaca, NY, USA.
54. Cho, K., Van Merriënboer, B., Bahdanau, D., and Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv.10.3115/v1/W14-4012 ↗
55. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
56. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv.
57. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
58. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL, USA.10.1109/CVPR.2009.5206848 ↗