Aroma Release of Olfactory Displays Based on Audio-Visual Content
Source abstract
Abstract
Variant approaches used to release scents in most recent olfactory displays rely on time for decision making. The applicability of such an approach is questionable in scenarios like video games or virtual reality applications, where the specific content is dynamic in nature and thus not known in advance. All of these are required to enhance the experience and involvement of the user while watching or participating virtually in 4D cinemas or fun parks, associated with short films. Recently, associating the release of scents to the visual content of the scenario has been studied. This research enhances one such work by considering the auditory content along with the visual content. Minecraft, a computer game, was used to collect the necessary dataset with 1200 audio segments. The Inception v3 model was used to classified the sound and image dataset. Further ground truth classification on this dataset resulted in four classes: grass, fire, thunder, and zombie. Higher accuracies of 91% and 94% were achieved using the transfer learning approach for the sound and image models, respectively.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Olfactory and Sensory Function Studies · Multisensory perception and integration · Virtual Reality Applications and Impacts
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer Science Department, King Abdulaziz University, Jeddah 21589, Saudi Arabia
Information Technology Department, King Abdulaziz University, Jeddah 21589, Saudi Arabia
College of Computing in Al-Qunfudhah, Umm Al Qura University, Makkah 24382, Saudi Arabia
References (28)
Show source-provided references
1. Tiny Olfactory Display Using Surface Acoustic Wave Device and Micropumps for Wearable Applications10.1109/JSEN.2016.2550486 ↗
2. Hashimoto, K., and Nakamoto, T. (2015, January 21–24). Stabilization of SAW atomizer for a wearable olfactory display. Proceedings of the 2015 IEEE International Ultrasonics Symposium (IUS), Taiwan, China.10.1109/ULTSYM.2015.0355 ↗
3. Defining virtual reality: Dimensions determining telepresence10.1111/j.1460-2466.1992.tb00812.x ↗
4. Kadowaki, A., Noguchi, D., Sugimoto, S., Bannai, Y., and Okada, K. (2010, January 19–23). Development of a High-Performance Olfactory Display and Measurement of Olfactory Characteristics for Pulse Ejections. Proceedings of the 2010 10th IEEE/IPSJ International Symposium on Applications and the Internet, Seoul, Korea.10.1109/SAINT.2010.39 ↗
5. Al Luhaybi, A., Alqurashi, F., Tsaramirsis, G., and Buhari, S.M. (2019). Automatic Association of Scents Based on Visual Content. Appl. Sci., 9.10.3390/app9081697 ↗
6. Valenti, M., Squartini, S., Diment, A., Parascandolo, G., and Virtanen, T. (2017, January 14–19). A convolutional neural network approach for acoustic scene classification. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.10.1109/IJCNN.2017.7966035 ↗
7. Tokozume, Y., and Harada, T. (2017, January 5–9). Learning environmental sounds with end-to-end convolutional neural network. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952651 ↗
8. Aytar, Y., Vondrick, C., and Torralba, A. (2016, January 5–10). Soundnet: Learning sound representations from unlabeled video. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain.10.1109/CVPR.2016.18 ↗
9. Dai, W., Dai, C., Qu, S., Li, J., and Das, S. (2017, January 5–9). Very deep convolutional neural networks for raw waveforms. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952190 ↗
10. Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification10.1109/LSP.2017.2657381 ↗
11. Kumar, A., Khadkevich, M., and Fügen, C. (2018, January 15–20). Knowledge Transfer from Weakly Labeled Audio Using Convolutional Neural Network for Sound Events and Scenes. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462200 ↗
12. AENet: Learning Deep Audio Features for Video Analysis10.1109/TMM.2017.2751969 ↗
13. Eghbal-zadeh, H., Lehner, B., Dorfer, M., and Widmer, G. (September, January 28). A hybrid approach with multi-channel i-vectors and convolutional neural networks for acoustic scene classification. Proceedings of the 2017 25th European Signal Processing Conference (EUSIPCO), Nairobi, Kenya.10.23919/EUSIPCO.2017.8081711 ↗
14. Zhang, Z., Xu, S., Cao, S., and Zhang, S. (2018, January 23–26). Deep Convolutional Neural Network with Mixup for Environmental Sound Classification. Proceedings of the Pattern Recognition and Computer Vision, Guangzhou, China.10.1007/978-3-030-03335-4_31 ↗
15. (2019, November 09). ESC: Dataset for Environmental Sound Classification. Available online: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/YDEPUT.
16. (2019, November 09). UrbanSound8k. Available online: https://urbansounddataset.weebly.com/urbansound8k.html.
17. Olfaction-Enhanced Multimedia: A Survey of Application Domains, Displays, and Research Challenges10.1145/2816454 ↗
18. Amores, J., and Maes, P. (2017, January 6–11). Essence: Olfactory Interfaces for Unconscious Influence of Mood and Cognitive Performance. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.10.1145/3025453.3026004 ↗
19. Smelling Screen: Development and Evaluation of an Olfactory Display System for Presenting a Virtual Odor Source10.1109/TVCG.2013.40 ↗
20. Dobbelstein, D., Herrdum, S., and Rukzio, E. (2017, January 11–15). inScent: A wearable olfactory display as an amplification for mobile notifications. Proceedings of the 2017 ACM International Symposium on Wearable Computers, Maui, Hawaii.10.1145/3123021.3123035 ↗
21. Development of a Portable, Multichannel Olfactory Display Transducer10.1109/JSEN.2018.2832284 ↗
22. (2019, April 06). Minecraft. Available online: https://minecraft.net/en-us/?ref=m.
23. (2019, March 28). Librosa. Available online: https://librosa.github.io/librosa/.
24. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the Inception Architecture for Computer Vision. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗
25. ImageNet Classification with Deep Convolutional Neural Networks10.1145/3065386 ↗
26. Simonyan, K., and Zisserman, A. (2019, June 25). Very Deep Convolutional Networks for Large-Scale Image Recognition. Available online: https://arxiv.org/abs/1409.1556.
27. (2019, March 20). ImageNet. Available online: http://www.image-net.org.
28. (2019, March 09). TensorFlow. Available online: https://www.tensorflow.org/.