journal article·2022Open access

Encoder-Decoder Based LSTM Model to Advance User QoE in 360-Degree Video

Muhammad Usman Younus, Rabia Shafi, Ammar Rafiq, Muhammad Rizwan Anjum, Sharjeel Afridi, Abdul Aleem Jamali, Zulfiqar Ali Arain

Computers, Materials & Continua· Volume 71· Issue 2

Abstract

The development of multimedia content has resulted in a massive increase in network traffic for video streaming. It demands such types of solutions that can be addressed to obtain the user's Quality-of-Experience (QoE). 360-degree videos have already taken up the user's behavior by storm. However, the users only focus on the part of 360-degree videos, known as a viewport. Despite the immense hype, 360-degree videos convey a loathsome side effect about viewport prediction, making viewers feel uncomfortable because user viewport needs to be pre-fetched in advance. Ideally, we can minimize the bandwidth consumption if we know what the user motion in advance. Looking into the problem definition, we propose an Encoder-Decoder based Long-Short Term Memory (LSTM) model to more accurately capture the non-linear relationship between past and future viewport positions. This model takes the transforming data instead of taking the direct input to predict the future user movement. Then, this prediction model is combined with a rate adaptation approach that assigns the bitrates to various tiles for 360-degree video frames under a given network capacity. Hence, our proposed work aims to facilitate improved system performance when QoE parameters are jointly optimized. Some experiments were carried out and compared with existing work to prove the performance of the proposed model. Last but not least, the experiments implementation of our proposed work provides high user's QoE than its competitors.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Image and Video Quality Assessment · Visual Attention and Saliency Detection · Video Coding and Compression Technologies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (30)

Show source-provided references
1. “360-degree camera market analysis–global forecast by 2018–2024.” 2020. [Online]. Available: https://www.marketwatch.com/press-release/360-degree-camera-market-size-volume-share-demand-growth-business-opportunity-by-2028-2021-06-21?tesla=y.
2. Rubiks: Practical 360-degree streaming for smartphones
3. Tile-based adaptive streaming for virtual reality video10.1145/3362101 ↗
4. A viewport prediction framework for panoramic videos
5. Plato: Learning-based adaptive streaming of 360-degree videos
6. 360probdash: Improving lSTM of 360 video streaming using tile-based http adaptive streaming
7. Adaptive 360-degree video streaming using scalable video coding
8. DRL360: 360-degree video streaming with deep reinforcement learning
9. PARIMA: Viewport adaptive 360-degree video streaming
10. Predicting head trajectories in 360 virtual reality videos
11. Your attention is unique: Detecting 360-degree video saliency in head-mounted display for head movement prediction
12. Mosaic: Advancing user quality of experience in 360-degree video streaming with machine learning10.1109/TNSM.2021.3053183 ↗
13. Predicting head movement in panoramic video: A deep reinforcement learning approach10.1109/TPAMI.2018.2858783 ↗
14. Optimizing the lifetime of software defined wireless sensor network via reinforcement learning10.1109/ACCESS.2020.3046693 ↗
15. Improving the software defined wireless sensor networks routing performance using reinforcement learning
16. RAPT360: Reinforcement learning-based rate adaptation for 360-degree video streaming with adaptive prediction and tiling
17. Younus · Contribution to energy optimization in WSN: Routing based on RL and SDN oriented routing · 2020
18. 360SRL: A sequential reinforcement learning approach for ABR tile-based 360 video streaming
19. Deep reinforcement learning-based rate adaptation for adaptive 360-degree video streaming
20. Video saliency prediction with optimized optical flow and gravity center bias
21. 360 video viewing dataset in head-mounted virtual reality
22. Shooting a moving target: Motion-prediction-based transmission for 360-degree videos
23. A hybrid algorithm of adaptive particle swarm optimization based on adaptive moment estimation method
24. Goodfellow · Deep Learning · 2016 · 1
25. Adaptive streaming of 360-degree videos with reinforcement learning
26. Flare: Practical viewport-adaptive 360-degree video streaming for mobile devices
27. 360 probdash: Improving QoE of 360 video streaming using tile-based http adaptive streaming
28. Optimizing 360 video delivery over cellular networks
29. 360-degree video streaming: A survey of the state of the art10.3390/sym12091491 ↗
30. MTC360: A multi-tiles configuration for viewport-dependent 360-gegree video streaming