“I’m a Fish!”: Exploring Children’s Engagement with Human–Data Interactions in Museums
Source abstract
Abstract
In an increasingly data-driven world, sparking children’s curiosity for meaningful data exploration provides a powerful foundation for lifelong data literacy. Human–data interaction (HDI) offers a promising approach by making data more accessible and engaging, particularly in informal learning environments like museums. However, there is limited understanding of how children, as a distinct user group, engage with embodied, interactive data visualizations. This paper presents findings from an exploratory field study of a gesture-controlled HDI installation deployed in a large urban museum. We analyzed the interactions of over 200 children, primarily from visiting K-12 school groups, as they engaged with an HDI prototype, the data on display, and each other. Our thematic analysis reveals that children’s interactions are deeply social, playful, and imaginative, often prioritizing collaborative discovery and role-playing over direct data interpretation. Based on these observations, we present design recommendations for creating HDI installations that leverage these behaviors to foster meaningful data engagement.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Museums and Cultural Heritage · Innovative Human-Technology Interaction · Participatory Visual Research Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Luddy School of Informatics, Computing and Engineering, Indiana University Indianapolis, Indianapolis, IN 46202, USA
Department of Computer Science, Faculty of Computers and Information Technology, University of Tabuk, Tabuk 47512, Saudi Arabia
Funding
Indiana University
References (82)
Show source-provided references
1. Exploring design principles for data literacy activities to support children’s inquiries from complex data10.1016/j.ijhcs.2019.03.006 ↗
2. Roberts, J., Lyons, L., Cafaro, F., and Eydt, R. (2014, January 17–20). Interpreting data from within: Supporting human-data interaction in museum exhibits through perspective taking. Proceedings of the ACM International Conference Proceeding Series, Aarhus Denmark.10.1145/2593968.2593974 ↗
3. Rogoff, B. (1990). Apprenticeship in Thinking: Cognitive Development in Social Context, Oxford University Press.10.1093/oso/9780195059731.001.0001 ↗
4. Research opportunities: Embodied child–computer interaction
5. The unintended effects of interactive objects and labels in the science museum
6. Museums as avenues of learning for children: A decade of research10.1007/s10984-016-9222-9 ↗
7. Eriksson, E., Baykal, G.E., and Torgersson, O. (2022, January 27–30). The role of learning theory in child-computer interaction—A semi-systematic literature review. Proceedings of the 21st Annual ACM Interaction Design and Children Conference, Braga, Portugal.10.1145/3501712.3529728 ↗
8. Cafaro, F. (2012, January 5–8). Using embodied allegories to design gesture suites for human-data interaction. Proceedings of the 2012 ACM Conference on Ubiquitous Computing—UbiComp’12, Pittsburgh, PA, USA.10.1145/2370216.2370309 ↗
9. Elmqvist, N. (2011, January 7–12). Embodied human-data interaction. Proceedings of the ACM CHI 2011 Workshop “Embodied Interaction: Theory and Practice in HCI, Vancouver, BC, Canada.
10. Cafaro, F., and Roberts, J. (2022). Data Through Movement: Designing Embodied Human-Data Interaction for Informal Learning, Springer Nature.10.1007/978-3-031-02610-2 ↗
11. Trajkova, M., Alhakamy, A., Cafaro, F., Mallappa, R., and Kankara, S.R. (2020, January 25–30). Move Your Body: Engaging Museum Visitors with Human-Data Interaction. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems—CHI ’20, Honolulu, HI, USA.10.1145/3313831.3376186 ↗
12. Mishra, S., and Cafaro, F. (2018, January 18–21). Full body interaction beyond fun: Engaging museum visitors in human-data interaction. Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction, Stockholm, Sweden.10.1145/3173225.3173291 ↗
13. Dourish, P. (2001). Where the Action Is, MIT Press.10.7551/mitpress/7221.001.0001 ↗
14. The role of physicality in tangible and embodied interactions10.1145/1925820.1925826 ↗
15. Perry, D.L. (2012). What Makes Learning Fun? Principles for the Design of Intrinsically Motivating Museum Exhibits, Rowman Altamira.
16. Enhancing learning and engagement through embodied interaction within a mixed reality simulation10.1016/j.compedu.2016.01.001 ↗
17. Exploring Mid-Air Hand Interaction in Data Visualization10.1109/TVCG.2023.3332647 ↗
18. Tangible data visualization of physical activity for children and adolescents: A qualitative study of temporal transition of experiences
19. Embodied science and mixed reality: How gesture and motion capture affect physics education10.1186/s41235-017-0060-9 ↗
20. Johnson-Glenberg, M.C., Yu, C.S.P., Liu, F., Amador, C., Bao, Y., Yu, S., and LiKamWa, R. (2023). Embodied mixed reality with passive haptics in STEM education: Randomized control study with chemistry titration. Front. Virtual Real., 4.10.3389/frvir.2023.1047833 ↗
21. Embodied immersive virtual reality to enhance the conceptual understanding of charged particles: A qualitative study
22. Making the body tangible: Elementary geometry learning through VR10.1007/s40751-020-00071-7 ↗
23. Tölgyessy, M., Dekan, M., and Chovanec, L. (2021). Skeleton tracking accuracy and precision evaluation of kinect v1, kinect v2, and the azure kinect. Appl. Sci., 11.10.3390/app11125756 ↗
24. Funken, M., and Hanne, T. (2025). Comparing Classification Algorithms to Recognize Selected Gestures Based on Microsoft Azure Kinect Joint Data. Information, 16.10.3390/info16050421 ↗
25. Popovici, D.M., Iordache, D., Comes, R., Neamțu, C.G.D., and Băutu, E. (2022). Interactive exploration of virtual heritage by means of natural gestures. Appl. Sci., 12.10.3390/app12094452 ↗
26. Mendoza, M.A.D., De La Hoz Franco, E., and Gómez, J.E.G. (2023). Technologies for the preservation of cultural heritage—A systematic review of the literature. Sustainability, 15.10.3390/su15021059 ↗
27. Mapping history: Orienting museum visitors across time and space10.1145/3199669 ↗
28. Müller, J., Walter, R., Bailly, G., Nischt, M., and Alt, F. (2012, January 5–10). Looking Glass: A Field Study on Noticing Interactivity of a Shop Window. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, USA.10.1145/2207676.2207718 ↗
29. Ackad, C., Tomitsch, M., and Kay, J. (2016, January 7–12). Skeletons and Silhouettes: Comparing User Representations at a Gesture-based Large Display. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems—CHI ’16, San Jose, CA, USA.10.1145/2858036.2858427 ↗
30. Mortier, R., Haddadi, H., Henderson, T., McAuley, D., and Crowcroft, J. (2025, October 21). Human-Data Interaction: The Human Face of the Data-Driven Society. Available online: https://ssrn.com/abstract=2508051.
31. Understanding human-data interaction: Literature review and recommendations for design10.1016/j.ijhcs.2019.09.004 ↗
32. The value of learning talk: Applying a novel dialogue scoring method to inform interaction design in an open-ended, embodied museum exhibit10.1007/s11412-017-9262-x ↗
33. Schauble, L., Gleason, M., Lehrer, R., Bartlett, K., Petrosino, A., Allen, A., Clinton, K., Ho, E., Jones, M., and Lee, Y.S. (2003). Supporting science learning in museums. Learning Conversations in Museums, Routledge.
34. Black, G. (2020). The Informal Museum Learning Experience. Museums and the Challenge of Change, Routledge.10.4324/9781003043010-16 ↗
35. Roberts, J., and Lyons, L. (2022). Examining spontaneous perspective taking and fluid self-to-data relationships in informal open-ended data exploration. Situating Data Science, Routledge.10.4324/9781003165354-3 ↗
36. Falk, J.H., and Dierking, L.D. (2016). The Museum Experience Revisited, Routledge.10.4324/9781315417851 ↗
37. Cafaro, F., Panella, A., Lyons, L., Roberts, J., and Radinsky, J. (2013, January 27). I see you there! Developing identity-preserving embodied interaction for museum exhibits. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, France.10.1145/2470654.2466252 ↗
38. Parker, C., Tomitsch, M., and Kay, J. (2018, January 8–12). Does the public still look at public displays? A field observation of public displays in the wild. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Singapore.10.1145/3214276 ↗
39. Parker, C., Tomitsch, M., Davies, N., Valkanova, N., and Kay, J. (2020, January 25–30). Foundations for Designing Public Interactive Displays that Provide Value to Users. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems—CHI ’20, Honolulu, HI, USA.10.1145/3313831.3376532 ↗
40. Müller, J., Wilmsmann, D., Exeler, J., Buzeck, M., Schmidt, A., Jay, T., and Krüger, A. (2009). Display blindness: The effect of expectations on attention towards digital signage. Pervasive Computing, Proceedings of the 7th International Conference, Pervasive 2009, Nara, Japan, 11–14 May 2009, Springer. Proceedings 7.10.1007/978-3-642-01516-8_1 ↗
41. Parra, G., Klerkx, J., and Duval, E. (2014, January 3–4). Understanding Engagement with Interactive Public Displays: An Awareness Campaign in the Wild. Proceedings of the International Symposium on Pervasive Displays—PerDis ’14, Copenhagen, Denmark.10.1145/2611009.2611020 ↗
42. Memarovic, N., Langheinrich, M., Alt, F., Elhart, I., Hosio, S., and Rubegni, E. (2012, January 15–17). Using Public Displays to Stimulate Passive Engagement, Active Engagement, and Discovery in Public Spaces. Proceedings of the Media Architecture Biennale Conference: Participati, Aarhus, Denmark.10.1145/2421076.2421086 ↗
43. Weber, D., Voit, A., Kollotzek, G., van der Vekens, L., Hepting, M., Alt, F., and Henze, N. (2018, January 21–26). PD notify: Investigating personal content on public displays. Proceedings of the Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.10.1145/3170427.3188475 ↗
44. Horn, M.S., Banerjee, A., Bar-El, D., and Wallace, I.H. (2020, January 21–24). Engaging families around museum exhibits: Comparing tangible and multi-touch interfaces. Proceedings of the Interaction Design and Children Conference—IDC ’20, London, UK.
45. Kruger, R., and Carpendale, M. (October, January 29). Orientation and Gesture on Horizontal Displays. Proceedings of the UbiComp 2002 Workshop on Collaboration with Interactive Walls and Tables, Citeseer, Göteborg, Sweden.
46. Alt, F., Shirazi, A.S., Kubitza, T., and Schmidt, A. (–2, January 27). Interaction techniques for creating and exchanging content with public displays. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, France.
47. Dalsgaard, P., Dindler, C., and Halskov, K. (2011). Understanding the dynamics of engaging interaction in public spaces. Human-Computer Interaction–INTERACT 2011, Proceedings of the 13th IFIP TC 13 International Conference, Lisbon, Portugal, 5–9 September 2011, Springer. Proceedings, Part II 13.10.1007/978-3-642-23771-3_17 ↗
48. Müller, J., Alt, F., Michelis, D., and Schmidt, A. (2010, January 25–29). Requirements and design space for interactive public displays. Proceedings of the 18th ACM International Conference on Multimedia, Firenze, Italy.10.1145/1873951.1874203 ↗
49. Guiding the design and implementation of interactive public displays in educational settings10.1007/s40692-023-00280-0 ↗
50. Mai, C., and Hußmann, H. (2018, January 18). The Audience Funnel for Head Mounted Displays in Public Environments. Proceedings of the 2018 IEEE 4th Workshop on Everyday Virtual Reality (WEVR), Virtual.
51. The audience funnel: Observations of gesture based interaction with multiple Large Displays in a City Center10.1080/10447318.2011.555299 ↗
52. Brignull, H., and Rogers, Y. (2003, January 1–5). Enticing People to Interact with Large Public Displays in Public Spaces. Proceedings of the Human-Computer Interaction INTERACT ’03: IFIP TC13 International Conference on Human-Computer Interaction, Zurich, Switzerland.
53. Wouters, N., Downs, J., Harrop, M., Cox, T., Oliveira, E., Webber, S., Vetere, F., and Vande Moere, A. (2016, January 4–8). Uncovering the honeypot effect: How audiences engage with public interactive systems. Proceedings of the 2016 ACM Conference on Designing Interactive Systems, Brisbane, QLD, Australia.10.1145/2901790.2901796 ↗
54. Tomitsch, M., Ackad, C., Dawson, O., Hespanhol, L., and Kay, J. (2014, January 3–4). Who cares about the content? An analysis of playful behaviour at a public display. Proceedings of the PerDis 2014—Proceedings: 3rd ACM International Symposium on Pervasive Displays 2014. Association for Computing Machinery, Copenhagen, Denmark.10.1145/2611009.2611016 ↗
55. Vygotsky, L.S. (1978). Mind in Society: The Development of Higher Psychological Processes, Harvard University Press.
56. Social learning spaces and student engagement10.1080/07294360.2010.512629 ↗
57. Remixing to design learning: Social media and peer-to-peer interaction
58. Learning through social interaction: Shared cognitive experience, negotiation strategies, and joint concept construction for young children10.1016/S0885-2006(05)80078-X ↗
59. Falk, J.H., and Dierking, L.D. (2018). Learning from Museums, Rowman & Littlefield.
60. Allen, S. (2003). Looking for learning in visitor talk: A methodological exploration. Learning Conversations in Museums, Routledge.
61. Shaffer, S.E. (2016). Engaging Young Children in Museums, Routledge.10.4324/9781315429571 ↗
62. Explain this, explore that: A study of parent–child interaction in a children’s museum10.1111/cdev.13232 ↗
63. Haden, C.A., Cohen, T., Uttal, D.H., and Marcus, M. (2015). Building learning: Narrating experiences in a children’s museum. Cognitive Development in Museum Settings, Routledge.
64. Children’s museum experiences: Identifying powerful mediators of learning10.1111/j.2151-6952.2002.tb00057.x ↗
65. Young children visiting museums: Exhibits, children and teachers co-author the journey10.1080/14733285.2018.1480750 ↗
66. Quinto Lima, S., Buraglia, G., Kam-Kwai, W., and Roberts, J. (May, January 26). Data Bias Recognition in Museum Settings: Framework Development and Contributing Factors. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan.10.1145/3706598.3714092 ↗
67. Alhakamy, A., Trajkova, M., and Cafaro, F. (July, January 28). Show Me How You Interact, I Will Tell You What You Think: Exploring the Effect of the Interaction Style on Users’ Sensemaking about Correlation and Causation in Data. Proceedings of the 2021 ACM Designing Interactive Systems Conference—DIS ’21, Virtual.10.1145/3461778.3462083 ↗
68. Bowman, R., Nadal, C., Morrissey, K., Thieme, A., and Doherty, G. (2023, January 23–28). Using thematic analysis in healthcare HCI at CHI: A scoping review. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, Hamburg, Germany.10.1145/3544548.3581203 ↗
69. Variations in Children’s and adults’ engagement with museum artifacts10.1080/10645578.2012.714997 ↗
70. Iversen, O.S., Smith, R.C., and Dindler, C. (2017, January 27–30). Child as Protagonist: Expanding the Role of Children in Participatory Design. Proceedings of the 2017 Conference on Interaction Design and Children—IDC ’17, Stanford, CA, USA.10.1145/3078072.3079725 ↗
71. Muravevskaia, E., Kuriappan, B., Markopoulos, P., Lekshmi, S., M, K., and Schrier, K. (2024, January 17–20). Exploring Empathic Design for Children Based on Role-Play Activities: Opportunities and Challenges within the Indian Context. Proceedings of the 23rd Annual ACM Interaction Design and Children Conference—IDC ’24, Delft, The Netherlands.10.1145/3628516.3659382 ↗
72. Young children’s understanding of the role that sensory experiences play in knowledge acquisition10.2307/1131493 ↗
73. Neural mirroring and social interaction: Motor system involvement during action observation relates to early peer cooperation10.1016/j.dcn.2017.01.001 ↗
74. Words in action: Investigating students’ language acquisition and emotional performance through embodied learning10.1080/17501229.2019.1607355 ↗
75. Macrine, S.L., and Fugate, J.M. (2022). Movement Matters: How Embodied Cognition Informs Teaching and Learning, MIT Press.10.7551/mitpress/13593.001.0001 ↗
76. Feltovich, P.J., Spiro, R.J., and Coulson, R.L. (2012). Learning, teaching, and testing for complex conceptual understanding. Test Theory for a New Generation of Tests, Routledge.
77. Attitudes and perceptions of middle school students toward competitive activities in physical education10.1123/jtpe.30.1.69 ↗
78. An exploration of young adolescents’ social achievement goals and social adjustment in middle school10.1037/0022-0663.100.3.672 ↗
79. Martinovic, D., Freiman, V., Lekule, C.S., and Yang, Y. (2018). The roles of digital literacy in social life of youth. Encyclopedia of Information Science and Technology, IGI Global. [4th ed.].10.4018/978-1-5225-2255-3.ch201 ↗
80. Nature of Interactions Among Young Children and Adult Caregivers in a Children’s Museum10.1007/s10643-013-0601-x ↗
81. Hurtienne, J., and Israel, J.H. (2007, January 15–17). Image schemas and their metaphorical extensions. Proceedings of the 1st International Conference on Tangible and Embedded Interaction—TEI ’07, Baton Rouge, LA, USA.10.1145/1226969.1226996 ↗
82. Follmer, S., Raffle, H., Go, J., Ballagas, R., and Ishii, H. (2010, January 9–12). Video play: Playful interactions in video conferencing for long-distance families with young children. Proceedings of the 9th International Conference on Interaction Design and Children, Barcelona, Spain.10.1145/1810543.1810550 ↗