On Alpha-Expansion-Based Graph-Cut Optimization for Decoder-Side Depth Estimation
Source abstract
Abstract
In order to achieve high realism an acceptable level of user experience in immersive videos, it is crucial to provide both the best possible quality of depth maps and minimize computational time. In this paper, we propose a new approach to the decoder-side depth estimation (DSDE) problem, which uses the hierarchical alpha-expansion algorithm with additional improvements for the estimation designed to be more robust to compressed video artifacts and limited computational resources. As shown by the experimental results, the proposal simultaneously results in reduction of computation time of the estimation process (by almost 40%) and an improvement of quality of estimated depth maps. The increased quality is demonstrated by more than 6% Bjøntegaard delta gain compared to the Moving Picture Experts Group (MPEG) immersive video state-of-the-art DSDE method.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Vision and Imaging · Video Coding and Compression Technologies · Image Processing Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute of Multimedia Telecommunications, Poznan University of Technology, 60-965 Poznań, Poland
Funding
Ministry of Science and Higher Education
References (43)
Show source-provided references
1. Content format and quality of experience in virtual reality
2. MPEG Immersive Video Coding Standard10.1109/JPROC.2021.3062590 ↗
3. VirtualCube: An Immersive 3D Video Communication System10.1109/TVCG.2022.3150512 ↗
4. Overview and Efficiency of Decoder-Side Depth Estimation in MPEG Immersive Video10.1109/TCSVT.2022.3162916 ↗
5. Overview of current development in depth map coding of 3D video and its future10.1049/iet-spr.2019.0063 ↗
6. Garus, P., Jung, J., Maugey, T., and Guillemot, C. (2019, January 8–10). Bypassing Depth Maps Transmission for Immersive Video Coding. Proceedings of the 2019 Picture Coding Symposium (PCS), Ningbo, China.10.1109/PCS48520.2019.8954543 ↗
7. Dziembowski, A., Domanski, M., Grzelka, A., Mieloch, D., Stankowski, J., and Wegner, K. (2016, January 18–20). The influence of a lossy compression on the quality of estimated depth maps. Proceedings of the 2016 International Workshop on Systems, Signals and Image Processing (IWSSIP), Bratislava, Slovakia.10.1109/IWSSIP.2016.7502730 ↗
8. The MPEG Immersive Video Standard—Current Status and Future Outlook10.1109/MMUL.2022.3175654 ↗
9. A New Approach to Decoder-Side Depth Estimation in Immersive Video Transmission10.1109/TBC.2023.3291133 ↗
10. Ravi, S.L., Milovanovic, M., Morin, L., and Henry, F. (2022, January 25–27). A Study of Conventional and Learning-Based Depth Estimators for Immersive Video Transmission. Proceedings of the 2022 IEEE 29th International Conference on Image, Video and Signal Processing (MMSP), Shanghai, China.10.1109/MMSP55362.2022.9948931 ↗
11. Kolmogorov, V., and Zabih, R. (June, January 28). Multi-camera Scene Reconstruction via Graph Cuts. Proceedings of the 7th European Conference on Computer Vision (ECCV), London, UK.
12. Rogge, S., Bonatto, D., Sancho, J., Salvador, R., Juarez, E., Munteanu, A., and Lafruit, G. (2019, January 18–20). MPEG-I Depth Estimation Reference Software. Proceedings of the 2019 International Conference on 3D Immersion (IC3D), Brussels, Belgium.10.1109/IC3D48390.2019.8975995 ↗
13. Multi-label Depth Estimation for Graph Cuts Stereo Problems10.1007/s10851-010-0212-8 ↗
14. Segmentation-based Method of Increasing the Depth Maps Temporal Consistency
15. Depth Map Estimation for Free-Viewpoint Television and Virtual Navigation10.1109/ACCESS.2019.2963487 ↗
16. IV-PSNR—The Objective Quality Metric for Immersive Video Applications10.1109/TCSVT.2022.3179575 ↗
17. Mieloch, D., Dziembowski, A., Grzelka, A., Stankiewicz, O., and Domański, M. (2017, January 10–14). Graph-based multiview depth estimation using segmentation. Proceedings of the 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), Hong Kong, China.10.1109/ICME.2017.8019532 ↗
18. Motion and Structure Information Based Adaptive Weighted Depth Video Estimation10.1109/TBC.2015.2437197 ↗
19. Fast approximate energy minimization via graph cuts10.1109/34.969114 ↗
20. Review of stereo vision algorithms and their suitability for resource-limited systems10.1007/s11554-012-0313-2 ↗
21. Multiframe stereo matching with edges, planes, and superpixels10.1016/j.imavis.2019.05.006 ↗
22. Adrian, D., Adam, G., Dawid, M., and Olgierd, S. (2016, January 11–13). Depth map upsampling and refinement for FTV systems. Proceedings of the 2016 International Conference on Signals and Electronic Systems (ICSES), Kraków, Poland.
23. (2023). Common Test Conditions for MPEG Immersive Video. Standard No. ISO/IEC JTC1/SC29/WG4 MPEG2023/N0406.
24. Bjøntegaard, G. (2001, January 2–4). Calculation of average PSNR differences between RD-Curves. Proceedings of the ITU-T VCEG Meeting, Austin, TX, USA.
25. (2018). 3DoF+ Test Sequence ClassroomVideo. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2018/M42415.
26. (2018). Technicolor 3DoF+ Test Materials. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2018/M42349.
27. (2020). New Test Content for Immersive Video—Nokia Chess. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2019/M50787.
28. (2021). A New Computer Graphics Scene, Guitarist, Suitable for MIV Edition-2. Standard No. ISO/IEC JTC1/SC29/WG04 MPEG2021/M58080.
29. (2021). [MIV] ERP Content Proposal for MIV ver.1 Verification Test. Standard No. ISO/IEC JTC1/SC29/WG11, MPEG2021/M58433.
30. (2018). Proposition of New Sequences for Windowed-6DoF Experiments on Compression, Synthesis and Depth Estimation. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2018/M43318.
31. (2021). [MIV] New Cadillac Content Proposal for Advanced MIV v2 Investigations. Standard No. ISO/IEC JTC1/SC29/WG4, MPEG2021/M57186.
32. (2021). Interdigital Mirror Content Proposal for Advanced MIV Investigations on Reflection. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2020/M55710.
33. (2020). InterdigitalFan Content Proposal for MIV. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG/M54732.
34. (2020). InterdigitalGroup Content Proposal. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2020/M54731.
35. (2021). [MIV] Dancing Sequence for Verification Tests. Standard No. ISO/IEC JTC1/SC29/WG4 MPEG2021/M57751.
36. (2017). Light Field Content from 16-Camera Rig. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2017/M40010.
37. (2021). MIV] Breakfast New Natural Content Proposal for MIV. Standard No. ISO/IEC JTC1/SC29/WG4 MPEG2021/M56730.
38. (2021). Barn New Natural Content Proposal for MIV. Standard No. ISO/IEC JTC1/SC29/WG4 MPEG2021/M56632.
39. (2018). Kermit Test Sequence for Windowed 6DoF Activities. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2018/M43748.
40. (2020). Natural Outdoor Test Sequences. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2019/M51598.
41. (2016). Multiview Test Video Sequences for Free Navigation Exploration Obtained Using Paris of Cameras. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2018/M38247.
42. (2021). [MIV] Undistorted CBA Basketball Test Sequence for MPEG-I Visual. Standard No. ISO/IEC JTC1/SC29/WG11 MPEG2021/M58500.
43. (2023). [MIV] New Natural Content—MartialArts. Standard No. ISO/IEC JTC1/SC29/WG04 MPEG2023/M61949.