Human-Centered Artificial Intelligence for Designing Accessible Cultural Heritage
Source abstract
Abstract
This paper reviews the literature concerning technology used for creating and delivering accessible museum and cultural heritage sites experiences. It highlights the importance of the delivery suited for everyone from different areas of expertise, namely interaction design, pedagogical and participatory design, and it presents how recent and future artificial intelligence (AI) developments can be used for this aim, i.e.,improving and widening online and in situ accessibility. From the literature review analysis, we articulate a conceptual framework that incorporates key elements that constitute museum and cultural heritage online experiences and how these elements are related to each other. Concrete opportunities for future directions empirical research for accessibility of cultural heritage contents are suggested and further discussed.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Multimodal Machine Learning Applications · Visual Attention and Saliency Detection · Human Pose and Action Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Polytech Nice Sophia, Campus SophiaTech, Université Côte d’Azur, 930 Route des Colles, 06410 Biot Sophia Antipolis, France
Computer Science and Systems Engineering Dept., Autonomous Systems and Robotics Lab, ENSTA, Institut Polytechnique Paris, INRIA Flowers Team, 91762 Paris, France
Department of Instructional Technology, University of Twente, 7522 NB Enschede, The Netherlands
Department of Information Engineering and Computer Science, University of Trento, 38122 Trento, Italy
References (100+)
Show source-provided references
1. Exhibiting interaction: Conduct and collaboration in museums and galleries10.1525/si.2001.24.2.189 ↗
2. Labadi, S. (2013). UNESCO, Cultural Heritage, and Outstanding Universal Value: Value-Based Analyses of the World Heritage and Intangible Cultural Heritage Conventions, Rowman & Littlefield.
3. Improving heritage impact assessment: An analytical critique of the ICOMOS guidelines10.1080/13527258.2018.1477057 ↗
4. Modeling visitors’ profiles: A study to investigate adaptation aspects for museum learning technologies10.1145/1841317.1841322 ↗
5. Integrating material and digital: A new way for cultural heritage10.1145/2486227.2486239 ↗
6. Dash, K., and Grohall, G. (2016). Economic Impact of Creating and Exhibiting 3D Objects for Blind And Visually Impaired People in Museums, Econ. Institut Wirtsch.
7. World blindness and visual impairment: Despite many successes, the problem is growing
8. Sensory: Please Touch the Art10.1080/00043125.2018.1389580 ↗
9. Arenghi, A., Garofolo, I., and Sormoen, O. (2016). Accessibility as a Key Enabling Knowledge for Enhancement of Cultural Heritage, FrancoAngeli.
10. Universal design, inclusive design, accessible design, design for all: Different concepts—One goal? On the concept of accessibility—Historical, methodological and philosophical aspects10.1007/s10209-014-0358-z ↗
11. Ten, A., Kaushik, P., Oudeyer, P.Y., and Gottlieb, J. (2020). Humans monitor learning progress in curiosity-driven exploration. PsyarXiv.10.31234/osf.io/d4wat ↗
12. Oudeyer, P.Y. (2018). Computational theories of curiosity-driven learning. arXiv.10.31234/osf.io/3p8f6 ↗
13. Portelas, R., Colas, C., Weng, L., Hofmann, K., and Oudeyer, P.Y. (2020). Automatic Curriculum Learning For Deep RL: A Short Survey. arXiv.10.24963/ijcai.2020/671 ↗
14. Alaimi, M., Law, E., Pantasdo, K.D., Oudeyer, P.Y., and Sauzeon, H. (2020, January 25–30). Pedagogical Agents for Fostering Question-Asking Skills in Children. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3313831.3376776 ↗
15. Arrieta, A.B., Díaz-Rodríguez, N., Ser, J.D., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., and Benjamins, R. (2019). Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI. Inf. Fusion.
16. Goebel, R., Chander, A., Holzinger, K., Lecue, F., Akata, Z., Stumpf, S., Kieseberg, P., and Holzinger, A. (2018). Explainable AI: The new 42?. International Cross-Domain Conference for Machine Learning and Knowledge Extraction, Springer.10.1007/978-3-319-99740-7_21 ↗
17. Guidance on the conduct of narrative synthesis in systematic reviews
18. Talk your way round—A speech interface to a virtual museum10.1080/09638280110111379 ↗
19. Caputo, F.M., Ciortan, I.M., Corsi, D., De Stefani, M., and Giachetti, A. (2006). Gestural Interaction and Navigation Techniques for Virtual Museum Experiences, AVI* CH.
20. Dulyan, A., and Edmonds, E. (2010, January 22–26). AUXie: Initial evaluation of a blind-accessible virtual museum tour. Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction, Melbourne, Australia.10.1145/1952222.1952280 ↗
21. Shi, T., and Liu, H. (2016, January 13–15). Supporting World War I heritage digitisation and presentation through user-centered web design. Proceedings of the 2016 SAI Computing Conference (SAI), London, UK.10.1109/SAI.2016.7556157 ↗
22. Schwartz, M., and Kautonen, H. (2015, January 2–7). The Challenges and Opportunities of Designing National Digital Services for Cross-Border Use. Proceedings of the International Conference of Design, User Experience, and Usability, Los Angeles, CA, USA.10.1007/978-3-319-20889-3_11 ↗
23. Sorce, S., Gentile, V., Oliveto, D., Barraco, R., Malizia, A., and Gentile, A. (2018, January 6–8). Exploring Usability and Accessibility of Avatar-based Touchless Gestural Interfaces for Autistic People. Proceedings of the 7th ACM International Symposium on Pervasive Displays, Munich, Germany.10.1145/3205873.3210705 ↗
24. Yoshino, K., Obata, K., and Tokuhisa, S. (2017, January 6–11). FLIPPIN’ Exploring a Paper-based Book UI Design in a Public Space. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.10.1145/3025453.3025981 ↗
25. Rapp, A., Marcengo, A., Geymonat, M., Simeoni, R., and Console, L. (2013, January 21–26). E-Inclusion as the next challenge for sustainable consumption. Proceedings of the International Conference on Universal Access in Human-Computer Interaction, Las Vegas, NV, USA.10.1007/978-3-642-39188-0_24 ↗
26. A mobile RFID-based tour system with instant microblogging10.1016/j.jcss.2010.02.011 ↗
27. Higuera, J.F.D., and Pernas, F.J.D. (1999, January 7–1). MINERVA: Multimedia on the Internet for virtual arts. Proceedings of the IEEE International Conference on Multimedia Computing and Systems, Florence, Italy.
28. Usaka, T., Yura, S., Fujimori, K., Mori, H., and Sakamuram, K. (1998, January 15–17). A multimedia MUD system for the digital museum. Proceedings of the 3rd Asia Pacific Computer Human Interaction (Cat. No. 98EX110), Shonan Village Center, Kanagawa, Japan.
29. TOURBOT and WebFAIR: Web-operated mobile robots for tele-presence in populated exhibitions10.1109/MRA.2005.1458329 ↗
30. Ng, M.K., Primatesta, S., Giuliano, L., Lupetti, M.L., Russo, L.O., Farulla, G.A., Indaco, M., Rosa, S., Germak, C., and Bona, B. (2015, January 21–23). A cloud robotics system for telepresence enabling mobility impaired people to enjoy the whole museum experience. Proceedings of the 2015 10th International Conference on Design & Technology of Integrated Systems in Nanoscale Era (DTIS), Napoli, Italy.10.1109/DTIS.2015.7127391 ↗
31. Partarakis, N., Klironomos, I., Antona, M., Margetis, G., Grammenos, D., and Stephanidis, C. (2016, January 17–22). Accessibility of cultural heritage exhibits. Proceedings of the International Conference on Universal Access in Human-Computer Interaction, Toronto, ON, Canada.10.1007/978-3-319-40244-4_43 ↗
32. Chalkia, E., Bekiaris, E., Panou, M., and Loukea, M. (2015, January 2–7). The Expansion of a Scheme About ACCESSIBILITY in Tourism at the Cultural Sector. Proceedings of the International Conference on Universal Access in Human-Computer Interaction, Los Angeles, CA, USA.10.1007/978-3-319-20687-5_3 ↗
33. Communicating cosmology with multisensory metaphorical experiences10.22323/2.19020801 ↗
34. Beyond virtual museums: Experiencing immersive virtual reality in real museums10.1016/j.culher.2010.04.001 ↗
35. Coelho, A., Cardoso, P., van Zeller, M., Santos, L., Raimundo, J., and Vaz, R. (2021, January 01). Gamifying the Museological Experience. Available online: https://www.semanticscholar.org/paper/Gamifying-the-Museological-Experience-Coelho-Zeller/21e5b997f9da380d123300552c5dcfe20814c076.
36. Frid, E., Lindetorp, H., Hansen, K.F., Elblaus, L., and Bresin, R. (2019, January 4–9). Sound Forest: Evaluation of an Accessible Multisensory Music Installation. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK.10.1145/3290605.3300907 ↗
37. Designing an interactive exhibitor for assisting blind and visually impaired visitors in tactile exploration of original museum pieces10.1016/j.procs.2018.10.076 ↗
38. Halbach, T., and Tjostheim, I. (2019, January 2–6). Towards Reliable Accessibility Assessments of Science Center Exhibits. Proceedings of the IFIP Conference on Human-Computer Interaction, Paphos, Cyprus.10.1007/978-3-030-29381-9_3 ↗
39. Fogli, D., and Arenghi, A. (June, January 29). ‘Design for All ‘Versus’ One-Size-Fits-All’: The Case of Cultural Heritage. Proceedings of the Fifth International Workshop on Cultures of Participation in the Digital Age—CoPDA 2018, Castiglione della Pescaia, Grosseto, Italy.
40. Cera, V., Origlia, A., Cutugno, F., and Campi, M. (2018). Semantically Annotated 3D Material Supporting the Design of Natural User Interfaces for Architectural Heritage, AVI* CH.
41. Ott, M., Dagnino, F.M., Pozzi, F., and Tavella, M. (2014, January 25–27). Widening access to Intangible Cultural Heritage: Towards the development of an innovative platform. Proceedings of the International Conference on Universal Access in Human-Computer Interaction, Heraklion, Crete, Greece.10.1007/978-3-319-07440-5_64 ↗
42. Song, M., Elias, T., Martinovic, I., Mueller-Wittig, W., and Chan, T.K. (2004, January 16–18). Digital heritage application as an edutainment tool. Proceedings of the 2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry, Singapore.10.1145/1044588.1044621 ↗
43. Capture, modeling, and recognition of expert technical gestures in wheel-throwing art of pottery10.1145/2627729 ↗
44. Museum experience design: A modern storytelling methodology
45. Callaway, C., Stock, O., and Dekoven, E. (2014). Experiments with Mobile Drama in an Instrumented Museum for Inducing Conversation in Small Groups. ACM Trans. Interact. Intell. Syst., 4.10.1145/2584250 ↗
46. Influencing participation in group brainstorming through ambient intelligence10.1080/10447318.2016.1136179 ↗
47. Evaluating an automated mediator for joint narratives in a conflict situation10.1080/0144929X.2019.1637940 ↗
48. Conole, G., and Fill, K. (2005). A learning design toolkit to create pedagogically effective learning activities. J. Interact. Media Educ.10.5334/2005-8 ↗
49. Colas, C., Karch, T., Lair, N., Dussoux, J.M., Moulin-Frier, C., Dominey, P.F., and Oudeyer, P.Y. (2020). Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration. arXiv.
50. Socially guided intrinsic motivation for robot learning of motor skills10.1007/s10514-013-9339-y ↗
51. Grbic, D., Palm, R.B., Najarro, E., Glanois, C., and Risi, S. (2020). EvoCraft: A New Challenge for Open-Endedness. arXiv.10.1007/978-3-030-72699-7_21 ↗
52. Open-Ended Learning: A Conceptual Framework Based on Representational Redescription10.3389/fnbot.2018.00059 ↗
53. Oudeyer, P. (2017). Autonomous development and learning in artificial intelligence and robotics: Scaling up deep learning to human-like learning. arXiv.10.1017/S0140525X17000243 ↗
54. Falk, J.H., and Dierking, L.D. (2012). Lifelong science learning for adults: The role of free-choice experiences. Second International Handbook of Science Education, Springer.10.1007/978-1-4020-9041-7_70 ↗
55. Formal lessons improve informal educational experiences: The influence of prior knowledge on student engagement10.1080/10645578.2017.1297134 ↗
56. ICT and the future of tourist management10.1108/JTF-12-2018-0072 ↗
57. For Me or Not For Me? Exploring Young Adults’ Museum Representations10.1080/01490400.2018.1550455 ↗
58. Family learning in object-based museums: The role of joint attention10.1080/10645578.2015.1079095 ↗
59. Hohenstein, J., and Moussouri, T. (2018). Museum Learning, Routledge.10.4324/9781315696447 ↗
60. Visualizing biological data in museums: Visitor learning with an interactive tree of life exhibit10.1002/tea.21318 ↗
61. Reasoning about Objects in a Natural History Museum: The Effect of Complexity of Questions on Object Labels10.1080/10645578.2020.1781485 ↗
62. Designing for data with ask Dr. Discovery: Design approaches for facilitating museum evaluation with real-time data mining10.1007/s10758-017-9313-4 ↗
63. Ask Dr. Discovery: The impact of a casual mobile game on visitor engagement with science museum content10.1007/s11423-019-09696-x ↗
64. Lakkaraju, H., Kleinberg, J., Leskovec, J., Ludwig, J., and Mullainathan, S. (2017). The Selective Labels Problem: Evaluating Algorithmic Predictions in the Presence of Unobservables. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM.10.1145/3097983.3098066 ↗
65. Marty, P.F., Sayre, S., and Fantoni, S.F. (2011). Personal digital collections: Involving users in the co-creation of digital cultural heritage. Handbook of Research on Technologies and Cultural Heritage: Applications and Environments, IGI Global.
66. My lost museum: User expectations and motivations for creating personal digital collections on museum websites10.1016/j.lisr.2010.11.003 ↗
67. Mapping the informal science education landscape: An exploratory study10.1177/0963662510393606 ↗
68. Simonsen, I.J., and Robertson, T. (2013). Participatory Design. Routledge International Handbook of Participatory Design, Routledge.10.4324/9780203108543 ↗
69. Transforming society by transforming technology: The science and politics of participatory design
70. Ulrich, W. (1983). Critical Heuristics of Social Planning: A New Approach to Practical Philosophy, Wiely.
71. Scandinavian design: On participation and skill
72. Ehn, P. (2006). Participation in Interaction Design: Actors and Artifacts in Interaction, Lawrence Erlbaum Associates.
73. User participation and democracy: A discussion of Scandinavian research on system development
74. Braa, J. (1996, January 13–15). Community-based participatory design in the Third World. Proceedings of the Participatory Design Conference, Palo Alto, CA, USA.
75. Participatory design for, with, and by communities
76. Le Dantec, C.A. (2016). Designing Publics, MIT Press.10.7551/mitpress/10513.001.0001 ↗
77. Small projects/large changes: Participatory design as an open participated process10.1080/15710882.2011.630472 ↗
78. The breakdown of the municipality as caring platform: Lessons for co-design and co-learning in the age of platform capitalism10.1080/15710882.2019.1631354 ↗
79. Repositioning CoDesign in the age of platform capitalism: From sharing to caring
80. Smith, R.C., and Kjærsgaard, M.G. (2014, January 6–10). Design anthropology in participatory design from ethnography to anthropological critique?. Proceedings of the 13th Participatory Design Conference: Short Papers, Industry Cases, Workshop Descriptions, Doctoral Consortium Papers, and Keynote Abstracts, Windhoek, Namibia.10.1145/2662155.2662209 ↗
81. Dindler, C., Iversen, O.S., Smith, R., and Veerasawmy, R. (2010, January 22–26). Participatory design at the museum: Inquiring into children’s everyday engagement in cultural heritage. Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction, Brisbane, Australia.10.1145/1952222.1952239 ↗
82. Infrastructuring Knowledge in Cultural Infrastructure: Informal Example of Participatory Design for Museum Exhibition10.4018/IJSKD.2015010102 ↗
83. Zuljevic, M., van de Weijer, M., and Carabelli, G. (2018, January 20–24). Walking, recording and collaborative mapping: How can we advance PD methodology by engaging with heritage?. Proceedings of the 15th Participatory Design Conference: Short Papers, Situated Actions, Workshops and Tutorial, Hasselt, Belgium.10.1145/3210604.3210623 ↗
84. Things we value10.1145/1897239.1897245 ↗
85. Ciolfi, L. (2013, January 21–25). The collaborative work of heritage: Open challenges for CSCW. Proceedings of the ECSCW 2013: 13th European Conference on Computer Supported Cooperative Work, Paphos, Cyprus.10.1007/978-1-4471-5346-7_5 ↗
86. Cumbo, B.J., Eriksson, E., and Iversen, O.S. (2019, January 3–5). The “Least-Adult” Role in Participatory Design with Children. Proceedings of the 31st Australian Conference on Human-Computer-Interaction, Perth, Australia.10.1145/3369457.3369464 ↗
87. González, S. (2017). Contested Markets, Contested Cities: Gentrification and Urban Justice in Retail Spaces, Routledge.10.4324/9781315440361 ↗
88. Ciolfi, L., Damala, A., Hornecker, E., Lechner, M., and Maye, L. (2017). Cultural Heritage Communities: Technologies and Challenges, Routledge.
89. Creating tangible interactions with cultural heritage: Lessons learned from a large scale, long term co-design project10.1080/15710882.2019.1596288 ↗
90. Ciolfi, L., and Petrelli, D. (2015, January 27–30). Studying a community of volunteers at a historic cemetery to inspire interaction concepts. Proceedings of the 7th International Conference on Communities and Technologies, Limerick, Ireland.10.1145/2768545.2768547 ↗
91. The construction of fictional space in participatory design practice10.1080/15710882.2010.493941 ↗
92. Beneath the ‘Digital Native’myth: Understanding young Australians’ online time use10.1177/1440783311434856 ↗
93. Díaz-Rodríguez, N., and Pisoni, G. (2020, January 14–17). Accessible Cultural Heritage through Explainable Artificial Intelligence. Proceedings of the 11th Workshop on Personalized Access to Cultural Heritage, Genova, Italy.10.1145/3386392.3399276 ↗
94. High Level Expert Group on Artificial Intelligence (2019). Ethics Guidelines for Trustworthy AI, European Commission. Technical Report.
95. A survey of methods for explaining black box models10.1145/3236009 ↗
96. Gunning, D. (2017). Explainable Artificial Intelligence (xAI), Defense Advanced Research Projects Agency (DARPA). Technical Report.
97. Mueller, S.T., Hoffman, R.R., Clancey, W.J., Emrey, A., and Klein, G. (2019). Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI. arXiv.
98. Heuillet, A., Couthouis, F., and Díaz-Rodríguez, N. (2020). Explainability in Deep Reinforcement Learning. Knowledge-Based Systems, Jones & Bartlett Publishers.10.1016/j.knosys.2020.106685 ↗
99. Zhao, Y., Wu, S., Reynolds, L., and Azenkot, S. (2018). The Effect of Computer-Generated Descriptions on Photo-Sharing Experiences of People with Visual Impairments. arXiv.10.1145/3134756 ↗
100. Rocchi, C., Stock, O., Zancanaro, M., Kruppa, M., and Krüger, A. (2004, January 13–16). The museum visit: Generating seamless personalized presentations on multiple devices. Proceedings of the 9th International Conference on Intelligent User Interfaces, Madeira, Portugal.10.1145/964442.964517 ↗