Design and Evaluation of a Web- and Mobile-Based Binaural Audio Platform for Cultural Heritage
Source abstract
Abstract
PlugSonic is a suite of web- and mobile-based applications for the curation and experience of 3D interactive soundscapes and sonic narratives in the cultural heritage context. It was developed as part of the PLUGGY EU project (Pluggable Social Platform for Heritage Awareness and Participation) and consists of two main applications: PlugSonic Sample, to edit and apply audio effects, and PlugSonic Soundscape, to create and experience 3D soundscapes for headphones playback. The audio processing within PlugSonic is based on the Web Audio API and the 3D Tune-In Toolkit, while the mobile exploration of soundscapes in a physical space is obtained using Apple’s ARKit. The main goal of PlugSonic is technology democratisation; PlugSonic users—whether cultural institutions or citizens—are all given the instruments needed to create, process and experience 3D soundscapes and sonic narratives; without the need for specific devices, external tools (software and/or hardware), specialised knowledge or custom development. The aims of this paper are to present the design and development choices, the user involvement processes as well as a final evaluation conducted with inexperienced users on three tasks (creation, curation and experience), demonstrating how PlugSonic is indeed a simple, effective, yet powerful tool.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Hearing Loss and Rehabilitation · Multisensory perception and integration
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Dyson School of Design Engineering, Imperial College London, London SW7 2DB, UK
Funding
Horizon 2020
References (51)
Show source-provided references
1. The Faro convention, a new paradigm for socially-and culturally-sustainable heritage action?
2. Russo, A. (2012). The rise of the media museum: Creating interactive cultural experiences through social media. Heritage and Social Media. Understanding Heritage in a Participatory Culture, Routledge.
3. Stuedahl, D., and Mörtberg, C. (2012). Heritage knowledge, social media, and the sustainability of the intangible. Heritage and Social Media. Understanding Heritage in a Participatory Culture, Routledge.
4. Lim, V., Frangakis, N., Tanco, L.M., and Picinali, L. (2018). PLUGGY: A pluggable social platform for cultural heritage awareness and participation. Advances in Digital Cultural Heritage, Springer. Available online: https://doi.org/10.1007/978-3-319-75789-6_9.10.1007/978-3-319-75789-6_9 ↗
5. Participatory communication with social media10.1111/j.2151-6952.2008.tb00292.x ↗
6. (2021, January 27). PLUGGY Project. Available online: https://www.pluggy-project.eu/.
7. (2021, January 27). Europeana. Available online: https://www.europeana.eu/portal/en.
8. Katz, B.F., Murphy, D., and Farina, A. (2020). The Past Has Ears (PHE): XR explorations of acoustic spaces as cultural heritage. Lecture Notes in Computer Science. Proceedings of the International Conference on Augmented Reality, Virtual Reality and Computer Graphics, Lecce, Italy, 7–10 September 2020, Springer. Available online: https://doi.org/10.1007/978-3-030-58468-9_7.10.1007/978-3-030-58468-9_7 ↗
9. Acoustics of historic spaces as a form of intangible cultural heritage10.1017/S0003598X00049139 ↗
10. Soundscape conservation10.1007/s10980-011-9635-x ↗
11. Kytö, M., Rémy, N., Uimonen, H., Acquier, F., Bérubé, G., Chelkoff, G., Said, N.G., Laroche, S., McOisans, J., and Tixier, N. (2021, January 27). European Acoustic Heritage. Available online: https://hal.archives-ouvertes.fr/hal-00993848.
12. Sonic interactions in virtual reality: State of the art, current challenges, and future directions10.1109/MCG.2018.193142628 ↗
13. Meelberg, V. (2018, January 27–31). Narrative sonic ambiances. Proceedings of the Euronoise 2018, Crete, Greece. Available online: http://hdl.handle.net/2066/192407.
14. Delle Monache, S., Rocchesso, D., Qi, J., Buechley, L., De Götzen, A., and Cestaro, D. (2012, January 19–22). Paper mechanisms for sonic interaction. Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, Kingston, ON, Canada.10.1145/2148131.2148146 ↗
15. Soundscape ecology: The science of sound in the landscape10.1525/bio.2011.61.3.6 ↗
16. Krakowsky, T. (2009). Sonic storytelling: Designing musical spaces. AdAge, Available online: https://adage.com/article/on-design/sonic-storytelling-designing-musical-spaces/138028/.
17. Theatrophone: The 19th-century iPod10.1016/S0262-4079(08)60113-X ↗
18. (2021, January 27). WebAudio API. Available online: https://www.w3.org/TR/webaudio/.
19. (2021, January 27). WebGL. Available online: https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API.
20. (2021, January 27). Web Audio API Specification Proposal. Available online: https://www.w3.org/2011/audio/drafts/1WD/WebAudio/.
21. Poirier-Quinot, D., and Katz, B.F. (2018, January 23–26). The Anaglyph binaural audio engine. Proceedings of the Audio Engineering Society Convention 144, Milan, Italy. Available online: http://www.aes.org/e-lib/browse.cfm?elib=19544.
22. Carpentier, T., Noisternig, M., and Warusfel, O. (October, January 25). Twenty years of Ircam Spat: Looking back, looking forward. Proceedings of the 41st International Computer Music Conference (ICMC), Denton, TX, USA. Available online: https://hal.archives-ouvertes.fr/hal-01247594.
23. Musil, T., Noisternig, M., and Höldrich, R. (2005, January 20–22). A library for realtime 3d binaural sound reproduction in pure data (pd). Proceedings of the 8th international conference on digital audio effects (DAFX-05), Madrid, Spain. Available online: http://dafx.de/paper-archive/2005/P_167.pdf.
24. Cuevas-Rodríguez, M., Picinali, L., González-Toledo, D., Garre, C., de la Rubia-Cuestas, E., Molina-Tanco, L., and Reyes-Lecuona, A. (2019). 3D Tune-In Toolkit: An open-source library for real-time binaural spatialisation. PLoS ONE, 14, Available online: https://doi.org/10.1371/journal.pone.0211899.10.1371/journal.pone.0211899 ↗
25. Personalization in cultural heritage: The road travelled and the one ahead10.1007/s11257-011-9104-x ↗
26. Not, E., and Zancanaro, M. (2021, January 27). Content adaptation for audio-based hypertexts in physical environments. Hypertext’98: Second Workshop on Adaptive Hypertext and Hypermedia, Available online: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.57.1908.
27. User-centred design of flexible hypermedia for a mobile guide: Reflections on the HyperAudio experience10.1007/s11257-005-8816-1 ↗
28. Benelli, G., Bianchi, A., Marti, P., Not, E., and Sennati, D. (1999, January 7–11). HIPS: Hyper-interaction within physical space. Proceedings of the IEEE International Conference on Multimedia Computing and Systems, Florence, Italy. Available online: https://doi.ieeecomputersociety.org/10.1109/MMCS.1999.778663.
29. LISTEN: A user-adaptive audio-augmented museum guide10.1007/s11257-008-9049-x ↗
30. Delerue, O., and Warusfel, O. (2002, January 15–17). Authoring of virtual sound scenes in the context of the Listen project. Proceedings of the Audio Engineering Society Conference, 22nd International Conference: Virtual, Synthetic, and Entertainment Audio, Espoo, Finland.
31. Zimmermann, A., Lorenz, A., and Birlinghoven, S. (2003, January 6–8). Listen: Contextualized presentation for audio-augmented environments. Proceedings of the 11th Workshop on Adaptivity and User Modeling in Interactive Systems, Bonn, Germany.
32. (2021, January 27). IRCAM Spatialisateur. Available online: http://forumnet.ircam.fr/product/spat-en/.
33. Vayanou, M., Katifori, A., Karvounis, M., Kourtis, V., Kyriakidi, M., Roussou, M., Tsangaris, M., Ioannidis, Y., Balet, O., and Prados, T. (2014). Authoring personalized interactive museum stories. Lecture Notes in Computer Science, Proceedings of the International Conference on Interactive Digital Storytelling, Singapore, 3–6 November 2014, Springer. Available online: https://doi.org/10.1007/978-3-319-12337-0_4.10.1007/978-3-319-12337-0_4 ↗
34. Pujol, L., Katifori, A., Vayanou, M., Roussou, M., Karvounis, M., Kyriakidi, M., Eleftheratou, S., and Ioannidis, Y. (2013, January 16–19). From personalization to adaptivity: Creating immersive visits through interactive digital storytelling at the acropolis museum. Proceedings of the 9th International Conference on Intelligent Environments, Athens, Greece. Available online: https://doi.org/10.3233/978-1-61499-286-8-541.
35. Mobile urban drama: Interactive storytelling in real world environments10.1080/13614568.2012.617842 ↗
36. (2021, January 27). Emotive. Available online: https://emotiveproject.eu/.
37. Engaging visitors of archaeological sites through “emotive” storytelling experiences: A pilot at the ancient agora of Athens
38. (2021, January 27). Arches. Available online: https://www.arches-project.eu/.
39. Geier, M., and Spors, S. (2012, January 22–25). Spatial audio with the soundscape renderer. Proceedings of the 27th Tonmeistertagung—VDT International Convention, Cologne, Germany. Available online: https://www.int.uni-rostock.de/fileadmin/user_upload/publications/spors/2012/Geier_TMT2012_SSR.pdf.
40. Vaananen, R. (2003, January 22–25). User interaction and authoring of 3D sound scenes in the Carrouso EU project. Proceedings of the Audio Engineering Society Convention 114, Amsterdam, The Netherlands.
41. (2021, January 27). Echoes. Available online: https://echoes.xyz/.
42. Comunità, M., Gerino, A., Lim, V., and Picinali, L. (2020). PlugSonic: A web- and mobile-based platform for binaural audio and sonic narratives. arXiv, Available online: https://arxiv.org/abs/2008.04638.
43. (2021, January 27). Apple AR Kit. Available online: https://developer.apple.com/augmented-reality/.
44. (2021, January 27). Faro Convention. Available online: https://www.coe.int/en/web/conventions/full-list/-/conventions/treaty/199.
45. Comunità, M., Gerino, A., Lim, V., and Picinali, L. (2019, January 27–29). Web-based binaural audio and sonic narratives for cultural heritage. Proceedings of the Audio Engineering Society Conference: 2019 AES International Conference on Immersive and Interactive Audio, York, UK. Available online: http://www.aes.org/e-lib/browse.cfm?elib=20435.
46. (2021, January 27). Voyager Golden Record, Available online: https://voyager.jpl.nasa.gov/golden-record.
47. The system usability scale: Past, present, and future10.1080/10447318.2018.1455307 ↗
48. Reichheld, F.F., and Covey, S.R. (2006). The Ultimate Question: Driving Good Profits and True Growth, Harvard Business School Press.
49. Çamcı, A., Lee, K., Roberts, C.J., and Forbes, A.G. (2017, January 22–25). INVISO: A cross-platform user interface for creating virtual sonic environments. Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, Québec City, QC, Canada. Available online: https://doi.org/10.1145/3126594.3126644.10.1145/3126594.3126644 ↗
50. Jillings, N., Man, B., Moffat, D., and Reiss, J.D. (August, January 26). Web audio evaluation tool: A browser-based listening test environment. Proceedings of the 12th International Conference on Sound and Music Computing, Maynooth, Ireland. Available online: https://doi.org/10.5281/zenodo.851157.
51. Kraft, S., and Zölzer, U. (2014, January 1–4). BeaqleJS: HTML5 and JavaScript based framework for the subjective evaluation of audio quality. Proceedings of the Linux Audio Conference, Karlsruhe, Germany. Available online: http://lac.linuxaudio.org/2014/papers/26.pdf.