Perception of Audio–Visual Synchronization in Olfactory-Enhanced 360-Degree Video
Source abstract
Abstract
This study examines the impact of olfactory stimuli on user experience (UX) metrics in 360-degree videos under varying levels of audio–visual (AV) skew. Subjective responses and questionnaire results revealed that scents helped stabilize enjoyment and artifact tolerance scores, particularly under severe AV skews, compared to non-olfactory conditions. However, the stationary nature of the scent delivery device decreased the intensity of olfactory stimuli, limiting their potential impact. Objective analyses highlighted a masking effect in 360-degree videos, where participant visual exploration reduced sensitivity to AV skews. Despite these challenges, olfactory stimuli demonstrated resilience to AV skews, suggesting their potential to buffer negative effects and enhance immersive experiences. However, they did not significantly improve overall video quality ratings. The study underscores the need for advances in olfactory display technology, such as head-mounted scent emitters and dynamic sensory integration, to enhance multimedia experiences.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Subtitles and Audiovisual Media
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Informatics, Federal University of Espirito Santo, Vitoria 29075-910, ES, Brazil
TAUCHI Research Center, Tampere University, 33014 Tampere, Finland
Department of Computer Science, College of Engineering, Design and Physical Sciences, Brunel University London, Uxbridge UB8 3PH, UK
College of Engineering and Digital Design, University of Kent, Kent CT2 7NX, UK
Funding
Coordination for the Improvement of Higher Education Personnel · National Council for Scientific and Technological Development · Fundação de Amparo à Pesquisa e Inovação do Espírito Santo · Research Council of Finland
References (34)
Show source-provided references
1. Korbar, B., Tran, D., and Torresani, L. (2018). Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization. arXiv.
2. (1994, January 14–19). Evaluation of multimedia synchronization techniques. Proceedings of the IEEE International Conference on Multimedia Computing and Systems, Boston, MA, USA.
3. Heyse, J., Carlier, S., Verhelst, E., Vander Linden, C., De Backere, F., and De Turck, F. (2022). From Patient to Musician: A Multi-Sensory Virtual Reality Rehabilitation Tool for Spatial Neglect. Appl. Sci., 12.10.3390/app12031242 ↗
4. Silveira, A., and Santos, C. (2022, January 22). Ongoing Challenges of Evaluating Mulsemedia QoE. Proceedings of the 2nd Workshop on Multisensory Experiences—SensoryX’22, Porto Alegre, Brazil.10.5753/sensoryx.2022.20005 ↗
5. Perceived synchronization of olfactory multimedia10.1109/TSMCA.2010.2041224 ↗
6. Murray, N., Qiao, Y., Lee, B., Karunakar, A.K., and Muntean, G.M. (March, January 28). Subjective Evaluation of Olfactory and Visual Media Synchronization. Proceedings of the MMSys ’13: 4th ACM Multimedia Systems Conference, New York, NY, USA.
7. On Perceived AV Synchronization in 360° Multimedia10.1109/MMUL.2024.3415116 ↗
8. Mulsemedia DIY: A survey of devices and a tutorial for building your own mulsemedia environment10.1145/3319853 ↗
9. Javerliat, C., Elst, P.P., Saive, A.L., Lavoué, G., and Baert, P. (December, January 29). Nebula: An Affordable Open-Source and Autonomous Olfactory Display for VR Headsets. Proceedings of the VRST ’22: 28th ACM Symposium on Virtual Reality Software and Technology, New York, NY, USA.
10. User quality of experience of mulsemedia applications10.1145/2661329 ↗
11. Covaci, A., Trestian, R., Saleme, E.B., Comsa, I.S., Assres, G., Santos, C.A., and Ghinea, G. (2019, January 21–25). 360 Mulsemedia: A way to improve subjective QoE in 360 videos. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.10.1145/3343031.3350954 ↗
12. Why is Multimedia Quality of Experience Assessment a Challenging Problem?10.1109/ACCESS.2019.2936470 ↗
13. Visual–Olfactory Interactions: Bimodal Facilitation and Impact on the Subjective Experience10.1093/chemse/bjy018 ↗
14. Olfaction-enhanced multimedia: Perspectives and challenges10.1007/s11042-010-0581-4 ↗
15. Evaluating user experience with physiological monitoring: A systematic literature review
16. Review on psychological stress detection using biosignals10.1109/TAFFC.2019.2927337 ↗
17. Biofeedback: An overview in the context of heart-brain medicine10.3949/ccjm.75.Suppl_2.S31 ↗
18. Mateos-García, N., Gil-González, A.B., Luis-Reboredo, A., and Pérez-Lancho, B. (2023). Driver stress detection from physiological signals by virtual reality simulator. Electronics, 12.10.3390/electronics12102179 ↗
19. Petrescu, L., Petrescu, C., Mitruț, O., Moise, G., Moldoveanu, A., Moldoveanu, F., and Leordeanu, M. (2020). Integrating biosignals measurement in virtual reality environments for anxiety detection. Sensors, 20.10.3390/s20247088 ↗
20. Ohata, M., and Tanaka, T. (2022, January 7–9). Estimation of Stress Level Based on Biosignals in Response to Emotional Stimuli. Proceedings of the 2022 IEEE 4th Global Conference on Life Sciences and Technologies (LifeTech), Osaka, Japan.10.1109/LifeTech53646.2022.9754934 ↗
21. Babaei, E., Tag, B., Dingler, T., and Velloso, E. (2021, January 8–13). A Critique of Electrodermal Activity Practices at CHI. Proceedings of the CHI ’21: 2021 CHI Conference on Human Factors in Computing Systems, Online.10.1145/3411764.3445370 ↗
22. Dillen, N., Ilievski, M., Law, E., Nacke, L.E., Czarnecki, K., and Schneider, O. (2020, January 25–30). Keep Calm and Ride Along: Passenger Comfort and Anxiety as Physiological Responses to Autonomous Driving Styles. Proceedings of the CHI ’20: 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3313831.3376247 ↗
23. Kan John, P., Zhu, X., Gedeon, T., and Zhu, W. (May, January 29). Evaluating Human Impressions of an Initiative-taking Robot. Proceedings of the CHI EA ’22: 2022 CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA.10.1145/3491101.3519633 ↗
24. A methodological framework to assess the accuracy of virtual reality hand-tracking systems: A case study with the Meta Quest 210.3758/s13428-022-02051-8 ↗
25. Measuring Heart Rate Variability Using Commercially Available Devices in Healthy Children: A Validity and Reliability Study
26. Seo, J., Laine, T.H., and Sohn, K.A. (2019). An Exploration of Machine Learning Methods for Robust Boredom Classification Using EEG and GSR Data. Sensors, 19.10.3390/s19204561 ↗
27. A Tutorial for Olfaction-Based Multisensorial Media Application Design and Evaluation10.1145/3108243 ↗
28. Saleme, E.a.B., and Santos, C.A.S. (2015, January 27–30). PlaySEM: A Platform for Rendering MulSeMedia Compatible with MPEG-V. Proceedings of the WebMedia ’15: 21st Brazilian Symposium on Multimedia and the Web, Manaus, Brazil.10.1145/2820426.2820450 ↗
29. Wireless EEG: A survey of systems and studies10.1016/j.neuroimage.2022.119774 ↗
30. Greene, J.M., and D’Oliveira, M. (1982). Learning to Use Statistical Tests in Psychology. Learning to Use Statistical Tests in Psychology, The Open University.
31. Scoping review of bioelectrical signals uses in videogames for evaluation purposes10.1109/ACCESS.2022.3213070 ↗
32. Do I Smell Coffee? The Tale of a 360° Mulsemedia Experience
33. Kosiński, J., Szklanny, K., Wieczorkowska, A., and Wichrowski, M. (2018, January 9–12). An Analysis of Game-Related Emotions Using EMOTIV EPOC. Proceedings of the 2018 Federated Conference on Computer Science and Information Systems (FedCSIS), Poznań, Poland.10.15439/2018F296 ↗
34. Murray, N., Qiao, Y., Lee, B., Muntean, G.M., and Karunakar, A.K. (2013, January 15–19). Age and gender influence on perceived olfactory & visual media synchronization. Proceedings of the 2013 IEEE International Conference on Multimedia and Expo (ICME), San Jose, CA, USA.10.1109/ICME.2013.6607467 ↗