On the Selection of Transmitted Views for Decoder-Side Depth Estimation
Source abstract
Abstract
The selection of optimal views for transmission is critical for the coding efficiency of the MPEG Immersive Video (MIV) profile of Decoder-Side Depth Estimation (DSDE). Standard approaches, which favor a uniform camera distribution, often fail in scenes with complex geometry, leading to decreased quality of depth estimation, and thus, reduced quality of virtual views presented to a viewer. This paper proposes an adaptive view selection method that analyzes the scene’s percentage of occluded regions. Based on this analysis, the encoder dynamically selects a transmission strategy: for scenes with a low occlusion ratio (smaller than 10%), a uniform layout is maintained to maximize spatial coverage; for scenes with a high occlusion ratio, the system switches to grouping cameras into stereo pairs, which are more robust for decreasing numbers of occlusions. Experiments conducted using the TMIV reference software demonstrated that this approach yields measurable quality gains (up to 2 dB BD-IVPSNR) for complex test sequences, such as MartialArts and Frog, without requiring any modifications to the decoder.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Coding and Compression Technologies · Advanced Vision and Imaging · Advanced Image Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute of Multimedia Telecommunications, Poznan University of Technology, 60-965 Poznań, Poland
Funding
Ministry of Science and Higher Education of the Republic of Poland
References (41)
Show source-provided references
1. Standardization Status of Immersive Video Coding10.1109/JETCAS.2019.2898948 ↗
2. (2020). MPEG Call for Proposals on 3DoF+Visual (Standard No. ISO/IEC JTC1/SC29/WG11MPEG/N18145).
3. (2017). Requirements on 6DoF (v1) (Standard No. ISO/IECJTC1/SC29/WG11 MPEG N17073).
4. Xiang, S., Yu, L., Liu, Q., and Xiong, Z. (2013, January 19–23). A gradient-based approach for interference cancelation in systems with multiple Kinect cameras. Proceedings of the 2013 IEEE International Symposium on Circuits and Systems (ISCAS), Beijing, China.
5. A free-viewpoint television system for horizontal virtual navigation10.1109/TMM.2018.2790162 ↗
6. Fachada, S., Bonatto, D., Schenkel, A., and Lafruit, G. (2018, January 3–5). Depth image based view synthesis with multiple reference views for virtual reality. Proceedings of the 3DTV Conference 2018, Stockholm, Sweden, Helsinki, Finland.10.1109/3DTV.2018.8478484 ↗
7. 3-D Video Representation Using Depth Maps10.1109/JPROC.2010.2091090 ↗
8. Overview of the High Efficiency Video Coding (HEVC) standard10.1109/TCSVT.2012.2221191 ↗
9. Overview of the Versatile Video Coding (VVC) standard and its applications10.1109/TCSVT.2021.3101953 ↗
10. Overview of the Multiview and 3D Extensions of High Efficiency Video Coding10.1109/TCSVT.2015.2477935 ↗
11. (2023). Verification Test Report of MPEG Immersive Video (Standard No. ISO/IEC JTC1/SC29/WG4 MPEG2023/N0341).
12. MPEG Immersive Video coding standard10.1109/JPROC.2021.3062590 ↗
13. Dziembowski, A., Samelak, J., and Domanski, M. (2018, January 10–12). View selection for virtual view synthesis in free navigation systems. Proceedings of the International Conference on Signals and Electronic Systems, ICSES, Kraków, Poland.10.1109/ICSES.2018.8507338 ↗
14. Dziembowski, A., Grzelka, A., Mieloch, D., Stankiewicz, O., Wegner, K., and Domanski, M. (2016, January 4–7). Multiview Synthesis–improved view synthesis for virtual navigation. Proceedings of the 32nd Picture Coding Symposium (PCS), Nuremberg, Germany.10.1109/PCS.2016.7906380 ↗
15. Robust multiview synthesis for wide-baseline camera arrays10.1109/TMM.2018.2802646 ↗
16. Hole filling with multiple reference views in DIBR view synthesis10.1109/TMM.2018.2791810 ↗
17. (2017). View Synthesis Reference Software (VSRS) 4.2 with Improved Inpainting and Hole Filling (Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2013/M40657).
18. Klóska, D., Dziembowski, A., and Samelak, J. (2023, January 15). Versatile input view selection for efficient immersive video transmission. Proceedings of the 31st International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, WSCG, Prague, Czechia.
19. (2018). Kermit Test Sequence for Windowed 6DoF Activities (Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2018/M43748).
20. Optimal camera placement for motion capture systems10.1109/TVCG.2016.2637334 ↗
21. An occlusion metric for selecting robust camera configurations10.1007/s00138-007-0094-y ↗
22. Optimal camera placement for accurate reconstruction10.1016/S0031-3203(01)00076-0 ↗
23. Qian, N., and Lo, C.-Y. (2015, January 14–15). Optimizing camera positions for multiview 3D reconstruction. Proceedings of the International Conference on 3D Imaging (IC3D), Liege, Belgium.
24. Gargallo, P., Prados, E., and Sturm, P. (2007, January 14–21). Minimizing the reprojection error in surface reconstruction from images. Proceedings of the IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Brazil.10.1109/ICCV.2007.4409003 ↗
25. Domański, M., Dziembowski, A., Grzelka, A., and Mieloch, D. (2016, January 5–7). Optimization of camera positions for free-navigation applications. Proceedings of the International Conference on Signals and Electronic Systems, ICSES 2016, Kraków, Poland.10.1109/ICSES.2016.7593833 ↗
26. Depth map coding for view synthesis based on distortion analyses10.1109/JETCAS.2014.2298314 ↗
27. Accelerating image-domain-warping virtual view synthesis on GPGPU10.1109/TMM.2017.2654120 ↗
28. FTV for 3-D spatial communication10.1109/JPROC.2011.2182101 ↗
29. (2013). Enhanced Depth Estimation Reference Software (DERS) for Freeviewpoint Television (Standard No. ISO/IEC JTC1/SC29/WG11 MPEG/M31518).
30. (2015). Depth Based View Blending in View Synthesis Reference Software (VSRS) (Standard No. ISO/IEC JTC1/SC29/WG11 MPEG/M37232).
31. Sun, Z., and Jung, C. (2015, January 17–19). Real-time depth-image-based rendering on GPU. Proceedings of the International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, Xi’an, China.10.1109/CyberC.2015.97 ↗
32. (2021). Manual of Immersive Video Depth Estimation (Standard No. ISO/IEC JTC1/SC29/WG04 MPEG VC/N0058).
33. (2024). Common Test Conditions for MPEG Immersive Video (Standard No. ISO/IEC JTC1/SC29/WG04 MPEG VC/N0539).
34. Samelak, J., Dziembowski, A., and Mieloch, D. (2022). Advanced HEVC Screen Content Coding for MPEG Immersive Video. Electronics, 11.10.3390/electronics11234040 ↗
35. (2021). Test Model 9 for MPEG Immersive Video (Standard No. ISO/IEC JTC1/SC29/WG4 MPEG2021/N0084).
36. Wieckowski, A., Brandenburg, J., Hinz, T., Bartnik, C., George, V., Hege, G., Helmrich, C., Henkel, A., Lehmann, C., and Stoffers, C. (2021, January 5–9). VVenC: An Open and Optimized VVC Encoder Implementation. Proceedings of the IEEE International Conference on Multimedia Expo Workshops (ICMEW), Online.10.1109/ICMEW53276.2021.9455944 ↗
37. Version [7.1]—[IV-PSNR: Software for immersive video objective quality evaluation]10.1016/j.softx.2024.101961 ↗
38. (2015). [FTV AHG] Big Buck Bunny Light-Field Test Sequences (Standard No. ISO/IEC JTC1/SC29/WG11 M35721).
39. (2014). FTV AHG: EE1 and EE2 Results with Bee by NICT Document IEC JTC1/SC29/WG11 (Standard No. MPEG2013/M32995).
40. (2008). 1D Parallel Test Sequences for MPEG-FTV (Standard No. ISO/IEC JTC1/SC29/WG11 M15378).
41. (2013). Enhanced View Synthesis Reference Software (VSRS) for Free-Viewpoint Television (Standard No. ISO/IEC JTC 1/SC 29/WG 11 M31520).