AI and Generative Models in 360-Degree Video Creation: Building the Future of Virtual Realities
Source abstract
Abstract
The generation of 360° video is gaining prominence in immersive media, virtual reality (VR), gaming projects, and the emerging metaverse. Traditional methods for panoramic content creation often rely on specialized hardware and dense video capture, which limits scalability and accessibility. Recent advances in generative artificial intelligence, particularly diffusion models and neural radiance fields (NeRFs), are examined in this research for their potential to generate immersive panoramic video content from minimal input, such as a sparse set of narrow-field-of-view (NFoV) images. To investigate this, a structured literature review of over 70 recent papers in panoramic image and video generation was conducted. We analyze key contributions from models such as 360DVD, Imagine360, and PanoDiff, focusing on their approaches to motion continuity, spatial realism, and conditional control. Our analysis highlights that achieving seamless motion continuity remains the primary challenge, as most current models struggle with temporal consistency when generating long sequences. Based on these findings, a research direction has been proposed that aims to generate 360° video from as few as 8–10 static NFoV inputs, drawing on techniques from image stitching, scene completion, and view bridging. This review also underscores the potential for creating scalable, data-efficient, and near-real-time panoramic video synthesis, while emphasizing the critical need to address temporal consistency for practical deployment.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Virtual Reality Applications and Impacts
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Technology, Yoobee College of Creative Innovation, Auckland 1010, New Zealand
Auckland Bioengineering Institute, University of Auckland, Auckland 1010, New Zealand
References (71)
Show source-provided references
1. Wong, E.S., Wahab, N.H.A., Saeed, F., and Alharbi, N. (2022). 360-degree video bandwidth reduction: Technique and approaches comprehensive review. Appl. Sci., 12.10.3390/app12157581 ↗
2. Shafi, R., Shuai, W., and Younus, M.U. (2020). 360-degree video streaming: A survey of the state of the art. Symmetry, 12.10.3390/sym12091491 ↗
3. Nerf: Representing scenes as neural radiance fields for view synthesis10.1145/3503250 ↗
4. Martin-Brualla, R., Radwan, N., Sajjadi, M.S., Barron, J.T., Dosovitskiy, A., and Duckworth, D. (2021, January 20–25). Nerf in the wild: Neural radiance fields for unconstrained photo collections. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00713 ↗
5. Barron, J.T., Mildenhall, B., Verbin, D., Srinivasan, P.P., and Hedman, P. (2022, January 18–24). Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.00539 ↗
6. Barron, J.T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., and Srinivasan, P.P. (2021, January 10–17). Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00580 ↗
7. Tan, J., Yang, S., Wu, T., He, J., Guo, Y., Liu, Z., and Lin, D. (2024). Imagine360: Immersive 360 Video Generation from Perspective Anchor. arXiv.
8. Wu, R., Chen, L., Yang, T., Guo, C., Li, C., and Zhang, X. (2023). Lamp: Learn a motion pattern for few-shot-based video generation. arXiv.10.1109/CVPR52733.2024.00677 ↗
9. Wen, Y. (2025). “See What I Imagine, Imagine What I See”: Human-AI Co-Creation System for 360° Panoramic Video Generation in VR. arXiv.
10. Ray, B., Jung, J., and Larabi, M.C. (2018, January 15–20). A low-complexity video encoder for equirectangular projected 360 video content. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462368 ↗
11. Park, M., Kang, T., Yun, J., Hwang, S., and Choo, J. (2025). SphereDiff: Tuning-free Omnidirectional Panoramic Image and Video Generation via Spherical Latent Representation. arXiv.
12. Feng, M., Liu, J., Cui, M., and Xie, X. (2023). Diffusion360: Seamless 360 degree panoramic image generation based on diffusion models. arXiv.
13. A survey on image and video stitching10.3724/SP.J.2096-5796.2018.0008 ↗
14. Yao, X., Hu, Q., Zhou, F., Liu, T., Mo, Z., Zhu, Z., Zhuge, Z., and Cheng, J. (2025). SpiNeRF: Direct-trained spiking neural networks for efficient neural radiance field rendering. Front. Neurosci., 19.10.3389/fnins.2025.1593580 ↗
15. Zhang, Q., Huang, C., Zhang, Q., Li, N., and Feng, W. (November, January 28). Learning geometry consistent neural radiance fields from sparse and unposed views. Proceedings of the 32nd ACM International Conference on Multimedia, Melbourne, VIC, Australia.
16. Zhang, Y., Zhang, G., Li, K., Zhu, Z., Wang, P., Wang, Z., Fu, C., Li, X., Fan, Z., and Zhao, Y. (2025). DASNeRF: Depth consistency optimization, adaptive sampling, and hierarchical structural fusion for sparse view neural radiance fields. PLoS ONE, 20.10.1371/journal.pone.0321878 ↗
17. Niemeyer, M., Barron, J.T., Mildenhall, B., Sajjadi, M.S., Geiger, A., and Radwan, N. (2022, January 18–24). Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00540 ↗
18. Younis, T., and Cheng, Z. (2025). Sparse-View 3D Reconstruction: Recent Advances and Open Challenges. arXiv.
19. Truong, P., Rakotosaona, M.J., Manhardt, F., and Tombari, F. (2023, January 17–24). Sparf: Neural radiance fields from sparse and noisy poses. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00408 ↗
20. Luo, C. (2022). Understanding diffusion models: A unified perspective. arXiv.
21. Generative AI: A systematic review using topic modelling techniques
22. Cheddak, A., Ait Baha, T., Es-Saady, Y., El Hajji, M., and Baslam, M. (2024). BERTopic for enhanced idea management and topic generation in Brainstorming Sessions. Information, 15.10.3390/info15060365 ↗
23. Sumantri, J.S., and Park, I.K. (2020, January 1–5). 360 panorama synthesis from a sparse set of images with unknown field of view. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass, CO, USA.10.1109/WACV45572.2020.9093582 ↗
24. Akimoto, N., Matsuo, Y., and Aoki, Y. (2022, January 18–24). Diverse plausible 360-degree image outpainting for efficient 3dcg background creation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01115 ↗
25. Cylin-Painting: Seamless 360 panoramic image outpainting and beyond10.1109/TIP.2023.3342570 ↗
26. Zheng, D., Zhang, C., Wu, X.M., Li, C., Lv, C., Hu, J.F., and Zheng, W.S. (2025). Panorama Generation From NFoV Image Done Right. arXiv.10.1109/CVPR52734.2025.02013 ↗
27. Nie, L., Lin, C., Liao, K., Liu, S., and Zhao, Y. (2022, January 18–24). Deep rectangling for image stitching: A learning baseline. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00565 ↗
28. Nie, L., Lin, C., Liao, K., Liu, S., and Zhao, Y. (2023, January 1–6). Parallax-tolerant unsupervised deep image stitching. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00680 ↗
29. Zhou, T., Li, H., Wang, Z., Luo, A., Zhang, C.L., Li, J., Zeng, B., and Liu, S. (2024, January 16–22). Recdiffusion: Rectangling for image stitching with diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR52733.2024.00260 ↗
30. Trepkowski, C., Eibich, D., Maiero, J., Marquardt, A., Kruijff, E., and Feiner, S. (2019, January 23–27). The effect of narrow field of view and information density on visual search performance in augmented reality. Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan.10.1109/VR.2019.8798312 ↗
31. Chen, Y., Zheng, C., Xu, H., Zhuang, B., Vedaldi, A., Cham, T.J., and Cai, J. (2024, January 10–15). MVSplat360: Feed-forward 360 scene synthesis from sparse views. Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Vancouver, BC, Canada.
32. Bao, C., Zhang, X., Yu, Z., Shi, J., Zhang, G., Peng, S., and Cui, Z. (2025). Free360: Layered Gaussian Splatting for Unbounded 360-Degree View Synthesis from Extremely Sparse and Unposed Views. arXiv.10.1109/CVPR52734.2025.01527 ↗
33. Liu, F., Sun, W., Wang, H., Wang, Y., Sun, H., Ye, J., Zhang, J., and Duan, Y. (2025). ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model. arXiv.
34. Nagoor Kani, B.R., Lee, H.Y., Tulyakov, S., and Tulsiani, S. (October, January 29). UpFusion: Novel View Diffusion from Unposed Sparse View Observations. Proceedings of the Computer Vision—ECCV 2024: 18th European Conference, Milan, Italy. Proceedings, Part LXXVI.10.1007/978-3-031-73116-7_11 ↗
35. Irshad, M.Z., Zakharov, S., Liu, K., Guizilini, V., Kollar, T., Gaidon, A., Kira, Z., and Ambrus, R. (2023, January 1–6). Neo 360: Neural fields for sparse view synthesis of outdoor scenes. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.00843 ↗
36. Chen, Z., Wu, C., Shen, Z., Zhao, C., Ye, W., Feng, H., Ding, E., and Zhang, S.H. (2025, January 10–17). Splatter-360: Generalizable 360° Gaussian Splatting for Wide-baseline Panoramic Images. Proceedings of the Computer Vision and Pattern Recognition Conference, Nashville, TN, USA.10.1109/CVPR52734.2025.02011 ↗
37. Dastjerdi, M.R.K., Hold-Geoffroy, Y., Eisenmann, J., Khodadadeh, S., and Lalonde, J.F. (2022, January 12–16). Guided co-modulated gan for 360° field of view extrapolation. Proceedings of the 2022 International Conference on 3D Vision (3DV), Prague, Czech Republic.10.1109/3DV57658.2022.00059 ↗
38. Zhang, C., Wu, Q., Gambardella, C.C., Huang, X., Phung, D., Ouyang, W., and Cai, J. (2024, January 16–22). Taming Stable Diffusion for Text to 360° Panorama Image Generation. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00607 ↗
39. Wang, H., Xiang, X., Fan, Y., and Xue, J.H. (2024, January 3–8). Customizing 360-degree panoramas through text-to-image diffusion models. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV57701.2024.00486 ↗
40. Zhou, D., Wang, W., Yan, H., Lv, W., Zhu, Y., and Feng, J. (2022). Magicvideo: Efficient video generation with latent diffusion models. arXiv.
41. An, J., Zhang, S., Yang, H., Gupta, S., Huang, J.B., Luo, J., and Yin, X. (2023). Latent-shift: Latent diffusion with temporal shift for efficient text-to-video generation. arXiv.
42. Chen, T.S., Lin, C.H., Tseng, H.Y., Lin, T.Y., and Yang, M.H. (2023). Motion-conditioned diffusion model for controllable video synthesis. arXiv.
43. Zhang, S., Wang, J., Zhang, Y., Zhao, K., Yuan, H., Qin, Z., Wang, X., Zhao, D., and Zhou, J. (2023). I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv.
44. Ni, H., Shi, C., Li, K., Huang, S.X., and Min, M.R. (2023, January 17–24). Conditional image-to-video generation with latent flow diffusion models. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01769 ↗
45. Wang, Q., Li, W., Mou, C., Cheng, X., and Zhang, J. (2024, January 16–22). 360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00660 ↗
46. LaMD: Latent Motion Diffusion for Image-Conditional Video Generation10.1007/s11263-025-02386-7 ↗
47. Unterthiner, T., van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., and Gelly, S. (2025, August 01). FVD: A New Metric for Video Generation. Available online: https://openreview.net/pdf?id=rylgEULtdN.
48. Unterthiner, T., Van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., and Gelly, S. (2018). Towards accurate generative models of video: A new metric & challenges. arXiv.
49. Hessel, J., Holtzman, A., Forbes, M., Bras, R.L., and Choi, Y. (2022). CLIPScore: A Reference-free Evaluation Metric for Image Captioning. arXiv.10.18653/v1/2021.emnlp-main.595 ↗
50. Xie, K., Sabour, A., Huang, J., Paschalidou, D., Klar, G., Iqbal, U., Fidler, S., and Zeng, X. (2025). VideoPanda: Video Panoramic Diffusion with Multi-view Attention. arXiv.
51. Xia, Y., Weng, S., Yang, S., Liu, J., Zhu, C., Teng, M., Jia, Z., Jiang, H., and Shi, B. (2025). PanoWan: Lifting Diffusion Video Generation Models to 360° with Latitude/Longitude-aware Mechanisms. arXiv.
52. Liu, J., Lin, S., Li, Y., and Yang, M.H. (2025, January 10–17). Dynamicscaler: Seamless and scalable video generation for panoramic scenes. Proceedings of the Computer Vision and Pattern Recognition Conference, Nashville, TN, USA.10.1109/CVPR52734.2025.00576 ↗
53. Cross-view panorama image synthesis10.1109/TMM.2022.3162474 ↗
54. Panogen: Text-conditioned panoramic environment generation for vision-and-language navigation
55. Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., and Feng, G. (2024). Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv.
56. Choi, D., Jang, H., and Kim, M.H. (2024, January 16–22). OmniLocalRF: Omnidirectional Local Radiance Fields from Dynamic Videos. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00656 ↗
57. Zhang, M., Chen, Y., Xu, R., Wang, C., Yang, J., Meng, W., Guo, J., Zhao, H., and Zhang, X. (March, January 20). PanoDiT: Panoramic Videos Generation with Diffusion Transformer. Proceedings of the AAAI Conference on Artificial Intelligence, Phfodelphia, PA, USA.
58. Xiong, Z., Chen, Z., Li, Z., Xu, Y., and Jacobs, N. (2025). PanoDreamer: Consistent Text to 360-Degree Scene Generation. arXiv.
59. Huo, Y., and Kuang, H. (2022, January 18–22). Ts360: A two-stage deep reinforcement learning system for 360-degree video streaming. Proceedings of the 2022 IEEE International Conference on Multimedia and Expo (ICME), Taipei, Taiwan.10.1109/ICME52920.2022.9859997 ↗
60. Yoon, Y., Chung, I., Wang, L., and Yoon, K.J. (2022, January 18–24). Spheresr: 360° image super-resolution with arbitrary projection via continuous spherical image representation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00559 ↗
61. Omnidirectional video super-resolution using deep learning10.1109/TMM.2023.3267294 ↗
62. Yang, S., Tan, J., Zhang, M., Wu, T., Li, Y., Wetzstein, G., Liu, Z., and Lin, D. (2024). Layerpano3d: Layered 3d panorama for hyper-immersive scene generation. arXiv.10.1145/3721238.3730643 ↗
63. OmniPlane: A Recolorable Representation for Dynamic Scenes in Omnidirectional Videos10.1109/TVCG.2025.3531763 ↗
64. Behravan, M., Matković, K., and Gračanin, D. (2025, January 27–29). Generative AI for context-aware 3D object creation using vision-language models in augmented reality. Proceedings of the 2025 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), Lisbon, Portugal.10.1109/AIxVR63409.2025.00018 ↗
65. Rey-Area, M., Yuan, M., and Richardt, C. (2022, January 18–24). 360monodepth: High-resolution 360deg monocular depth estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.00374 ↗
66. Spherical image generation from a few normal-field-of-view images by considering scene symmetry
67. Lu, C.N., Chang, Y.C., and Chiu, W.C. (2021, January 20–25). Bridging the visual gap: Wide-range image blending. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00090 ↗
68. A survey on optimizing mobile delivery of 360 videos: Edge caching and multicasting10.1109/ACCESS.2023.3292335 ↗
69. A Systematic Literature Review on 360° Panoramic Applications in Architecture, Engineering, and Construction (AEC) Industry
70. Partarakis, N., and Zabulis, X. (2024). A review of immersive technologies, knowledge representation, and AI for human-centered digital experiences. Electronics, 13.10.3390/electronics13020269 ↗
71. Ai-empowered persuasive video generation: A survey