A Model-Based Approach of Foreground Region of Interest Detection for Video Codecs
Source abstract
Abstract
Detecting the Region of Interest (ROI) for video clips is a significant and useful technique both in video codecs and surveillance/monitor systems. In this paper, a new model-based detection method is designed which suits video compression codecs by proposing two models: an “inter” and “intra” model. The “inter” model exploits the motion information represented as blocks by global motion compensation approaches while the “intra” model extracts the objects details through objects filtering and image segmentation procedures. Finally, the detection results are formed through a new clustering with fine-tune approach from the “intra” model assisted with the “inter” model. Experimental results show that the proposed method fits well for real-time video codecs and it achieves a good performance both on detection precision and on computing time. In addition, the proposed method is versatile for a wide range of surveillance videos with different characteristics.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Advanced Image and Video Retrieval Techniques · Advanced Vision and Imaging
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institution of Electronic Information Engineering, Beijing Jiaotong University, Shang Yuan Road No. 3, Haidian District, Beijing 100044, China
Artificial Intelligence Research Institute of China Unicom, Beijing 100048, China
References (73)
Show source-provided references
1. Digital color image processing within the framework of a human visual model10.1109/TASSP.1979.1163262 ↗
2. A region-based H.263+ codec and its rate control for low VBR video10.1109/TMM.2004.827488 ↗
3. Tong, L., and Rao, K. (2005, January 13–16). Region of interest based H.263 compatible codec and its rate control for low bit rate video conferencing. Proceedings of the 2005 International Symposium on Intelligent Signal Processing and Communication Systems, Hong Kong, China.
4. Mukherjee, D., Bankoski, J., Grange, A., Han, J., Koleszar, J., Wilkins, P., Xu, Y., and Bultje, R. (2013, January 8–11). The latest open-source video codec VP9 - An overview and preliminary results. Proceedings of the Picture Coding Symposium (PCS), San Jose, CA, USA.10.1109/PCS.2013.6737765 ↗
5. Overview of the High Efficiency Video Coding (HEVC) Standard10.1109/TCSVT.2012.2221191 ↗
6. A highly efficient system for automatic face region detection in MPEG video10.1109/76.611173 ↗
7. Object-oriented H.263 compatible video coding platform for conferencing applications10.1109/49.650919 ↗
8. Low bit-rate coding of image sequences using adaptive regions of interest10.1109/76.736718 ↗
9. Lam, C.F., and Lee, M. (1998). Video segmentation using color difference histogram. Multimedia Information Analysis and Retrieval, Proceedings of the IAPR International Workshop, MINAR’ 98, Hong Kong, China, 13–14 August 1998, Springer.10.1007/BFb0016496 ↗
10. MPEG-4: A multimedia compression standard for interactive applications and services10.1049/ecej:19980602 ↗
11. Zhou, X., Yang, C., Zhao, H., and Yu, W. (2014). Low-Rank Modeling and Its Applications in Image Analysis. CoRR.10.1145/2674559 ↗
12. Gupta, S., Davidson, J., Levine, S., Sukthankar, R., and Malik, J. (2017, January 21–26). Cognitive Mapping and Planning for Visual Navigation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.769 ↗
13. A New Rate Control Scheme For Video Coding Based On Region Of Interest10.1109/ACCESS.2017.2676125 ↗
14. Flow-Process Foreground Region of Interest Detection Method for Video Codecs10.1109/ACCESS.2017.2733219 ↗
15. ROI-Based Video Transmission in Heterogeneous Wireless Networks With Multi-Homed Terminals10.1109/ACCESS.2017.2748138 ↗
16. A low cost matching motion estimation sensor based on the NIOS II microprocessor10.3390/s121013126 ↗
17. Acceleration of block-matching algorithms using a custom instruction-based paradigm on a Nios II microprocessor10.1186/1687-6180-2013-118 ↗
18. Cogeneration of Fast Motion Estimation Processors and Algorithms for Advanced Video Coding10.1109/TVLSI.2010.2104166 ↗
19. Stauffer, C., and Grimson, W.E.L. (1999, January 23–25). Adaptive background mixture models for real-time tracking. Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), Fort Collins, CO, USA.
20. Learning Patterns of Activity Using Real-Time Tracking10.1109/34.868677 ↗
21. Zivkovic, Z. (2004, January 26–26). Improved adaptive Gaussian mixture model for background subtraction. Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK.10.1109/ICPR.2004.1333992 ↗
22. Cand, E., Li, X., Ma, Y., and Wright, J. (2010, January 4–7). Robust principal component analysis?: Recovering low-rank matrices from sparse errors. Proceedings of the 2010 IEEE Sensor Array and Multichannel Signal Processing Workshop, Jerusalem, Israel.
23. Block-Sparse RPCA for Salient Motion Detection10.1109/TPAMI.2014.2314663 ↗
24. Guyon, C., Bouwmans, T., and Zahzah, E.H. (October, January 30). Foreground detection based on low-rank and block-sparse matrix decomposition. Proceedings of the 2012 19th IEEE International Conference on Image Processing (ICIP), Orlando, FL, USA.10.1109/ICIP.2012.6467087 ↗
25. Robust PCA via Outlier Pursuit10.1109/TIT.2011.2173156 ↗
26. Sparse Bayesian Methods for Low-Rank Matrix Estimation10.1109/TSP.2012.2197748 ↗
27. Balzano, L., Nowak, R., and Recht, B. (October, January 29). Online identification and tracking of subspaces from highly incomplete information. Proceedings of the 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, VA, USA.10.1109/ALLERTON.2010.5706976 ↗
28. RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images10.1109/TPAMI.2011.282 ↗
29. Zhou, T., and Tao, D. (July, January 28). GoDec: Randomized Low-rank and Sparse Matrix Decomposition in Noisy Case. Proceedings of the 28th ICML, Bellevue, WA, USA.
30. Layered Dynamic Textures10.1109/TPAMI.2009.110 ↗
31. Modeling, Clustering, and Segmenting Video with Mixtures of Dynamic Textures10.1109/TPAMI.2007.70738 ↗
32. Clustering Dynamic Textures with the Hierarchical EM Algorithm for Modeling Video10.1109/TPAMI.2012.236 ↗
33. Generalized Stauffer–Grimson background subtraction for dynamic scenes10.1007/s00138-010-0262-3 ↗
34. ViBe: A Universal Background Subtraction Algorithm for Video Sequences10.1109/TIP.2010.2101613 ↗
35. Yao, J., and Odobez, J.M. (2007, January 17–22). Multi-Layer Background Subtraction Based on Color and Texture. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.10.1109/CVPR.2007.383497 ↗
36. A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications10.1109/TIP.2008.924285 ↗
37. Detection of Moving Objects Using Fuzzy Color Difference Histogram Based Background Subtraction10.1109/LSP.2015.2498839 ↗
38. Detection of Moving Objects Using Multi-channel Kernel Fuzzy Correlogram Based Background Subtraction10.1109/TCYB.2013.2274330 ↗
39. Fast approximate energy minimization via graph cuts10.1109/34.969114 ↗
40. What energy functions can be minimized via graph cuts?10.1109/TPAMI.2004.1262177 ↗
41. An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision10.1109/TPAMI.2004.60 ↗
42. Tang, M., Gorelick, L., Veksler, O., and Boykov, Y. (2013, January 1–8). GrabCut in One Cut. Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.222 ↗
43. He, K., Zhang, X., Ren, S., and Sun, J. (2014, January 6–12). Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10578-9_23 ↗
44. Deep Neural Networks for Object Detection
45. Erhan, D., Szegedy, C., Toshev, A., and Anguelov, D. (2014, January 24–27). Scalable Object Detection Using Deep Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.276 ↗
46. Szegedy, C., Reed, S.E., Erhan, D., and Anguelov, D. (2014). Scalable, High-Quality Object Detection. CoRR.
47. Ren, S., He, K., Girshick, R.B., and Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. CoRR.
48. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition10.1109/TPAMI.2015.2389824 ↗
49. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
50. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S.E., Fu, C., and Berg, A.C. (2015). SSD: Single Shot MultiBox Detector. CoRR.10.1007/978-3-319-46448-0_2 ↗
51. Huang, G., Liu, Z., and Weinberger, K.Q. (2016). Densely Connected Convolutional Networks. CoRR.10.1109/CVPR.2017.243 ↗
52. Dai, J., Li, Y., He, K., and Sun, J. (2016). R-FCN: Object Detection via Region-based Fully Convolutional Networks. CoRR.
53. Pinheiro, P.H.O., Collobert, R., and Dollár, P. (2015). Learning to Segment Object Candidates. CoRR.
54. Girshick, R.B., Donahue, J., Darrell, T., and Malik, J. (2013). Rich feature hierarchies for accurate object detection and semantic segmentation. CoRR.10.1109/CVPR.2014.81 ↗
55. Mining visual collocation patterns via self-supervised subspace learning10.1109/TSMCB.2011.2172605 ↗
56. Zhang, Z., Jing, T., Tian, C., Cui, P., Li, X., and Gao, M. (2019). Objects Discovery Based on Co-Occurrence Word Model with Anchor-Box Polishing. IEEE Trans. Circuits Syst. Video Technol.10.1109/TCSVT.2019.2894363 ↗
57. A Joint Approach to Global Motion Estimation and Motion Segmentation From a Coarsely Sampled Motion Vector Field10.1109/TCSVT.2011.2148490 ↗
58. Unger, M., Asbach, M., and Hosten, P. (2008, January 12–15). Enhanced background subtraction using global motion compensation and mosaicing. Proceedings of the 2008 15th IEEE International Conference on Image Processing, San Diego, CA, USA.10.1109/ICIP.2008.4712353 ↗
59. Jin, Y., Tao, L., Di, H., Rao, N.I., and Xu, G. (2008, January 12–15). Background modeling from a free-moving camera by Multi-Layer Homography Algorithm. Proceedings of the 2008 15th IEEE International Conference on Image Processing, San Diego, CA, USA.
60. Guerreiro, R.F.C., and Aguiar, P.M.Q. (2006). Global Motion Estimation: Feature-Based, Featureless, or Both?!. Image Analysis and Recognition, Proceedings of the Third International Conference, ICIAR 2006, voa de Varzim, Portugal, 18–20 September 2006, Springer.10.1007/11867586_66 ↗
61. (2019, March 25). The Change Detection Dataset. Available online: http://changedetection.net.
62. (2019, March 25). The HEVC/AVC Official Test Sequences, Online. Available online: ftp://hevc@ftp.tnt.uni-hannover.de/testsequences.
63. Global motion estimation from coarsely sampled motion vector field and the applications10.1109/TCSVT.2004.841656 ↗
64. Smolic, A., Hoeynck, M., and Ohm, J.R. (2000, January 10–13). Low-complexity global motion estimation from P-frame motion vectors for MPEG-7 applications. Proceedings of the 2000 International Conference on Image Processing (Cat. No.00CH37101), Vancouver, BC, Canada.10.1109/ICIP.2000.899352 ↗
65. Fischler, M.A., and Bolles, R.C. (1987). Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Readings in Computer Vision: Issues, Problems, Principles, and Paradigms, Morgan Kaufmann Publishers Inc.10.1016/B978-0-08-051581-6.50070-2 ↗
66. On the identification of the convex hull of a finite set of points in the plane10.1016/0020-0190(73)90020-3 ↗
67. Moving Object Detection by Detecting Contiguous Outliers in the Low-Rank Representation10.1109/TPAMI.2012.132 ↗
68. Rueda, L., Mery, D., and Kittler, J. (2007). Σ-Δ Background Subtraction and the Zipf Law. Progress in Pattern Recognition, Image Analysis and Applications, Proceedings of the 12th Iberoamericann Congress on Pattern Recognition, CIARP 2007, Valparaiso, Chile, 13–16 November 2007, Springer.10.1007/978-3-540-76725-1 ↗
69. Yokoyama, T., Iwasaki, T., and Watanabe, T. (2009, January 3–5). Motion Vector Based Moving Object Detection and Tracking in the MPEG Compressed Domain. Proceedings of the 2009 Seventh International Workshop on Content-Based Multimedia Indexing, Chania, Greece.10.1109/CBMI.2009.33 ↗
70. Yoneyama, A., Nakajima, Y., Yanagihara, H., and Sugano, M. (1999, January 24–28). Moving object detection and identification from MPEG coded data. Proceedings of the 1999 International Conference on Image Processing (Cat. 99CH36348), Kobe, Japan.10.1109/ICIP.1999.823035 ↗
71. Niu, C., and Liu, Y. (2009, January 17–19). Moving Object Segmentation Based on Video Coding Information in H.264 Compressed Domain. Proceedings of the 2009 2nd International Congress on Image and Signal Processing, Tianjin, China.10.1109/CISP.2009.5303718 ↗
72. Zhang, S., Su, X., and Xie, L. (2010, January 22–24). Global motion compensation for image sequences and motion object detection. Proceedings of the 2010 International Conference on Computer Application and System Modeling (ICCASM 2010), Taiyuan, China.
73. Gul, S., Meyer, J.T., Hellge, C., Schierl, T., and Samek, W. (2016, January 21–23). Hybrid video object tracking in H.265/HEVC video streams. Proceedings of the 2016 IEEE 18th International Workshop on Multimedia Signal Processing (MMSP), Montreal, QC, Canada.10.1109/MMSP.2016.7813363 ↗