journal article·2025-04-11Open access

MultiScaleAnalyzer for Spatiotemporal Learning Data Analysis: A Case Study of Eye-Tracking and Mouse Movement

Shuang Wei, Chen Guo, Qingli Lei, Yingjie Chen, Yan Ping Xin

Applied Sciences· Volume 15· Issue 8

Abstract

With the development of high-performance computers, cloud storage, and advanced sensors, people’s ability to gather complex learning data has greatly improved. However, analyzing these data remains a significant challenge. Especially for spatiotemporal learning data such as eye-tracking and mouse movement, understanding and analyzing these data to identify the learning insights behind them is a difficult task. We propose a visualization platform called “MultiScaleAnalyzer”, which employs hierarchical structure to illustrate spatiotemporal learning data in multiple views. From high-level overviews to detailed analyses, “MultiScaleAnalyzer” provides varying resolutions of data tailored to educators’ need. To demonstrate the platform’s effectiveness, we applied “MultiScaleAnalyzer” to a mathematical word problem-solving dataset, showcasing how the visualization platform facilitates the exploration of student problem-solving patterns and strategies.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Gaze Tracking and Assistive Technology · Data Visualization and Analytics · Image Retrieval and Classification Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Visual Communication Design, Jiangnan University, Wuxi 214122, China

School of Media Arts & Design, James Madison University, Harrisonburg, VA 22807, USA

Department of Special Education, University of Illinois Chicago, Chicago, IL 60607, USA

Department of Computer Graphics Technology, Purdue University, West Lafayette, IN 47907, USA

Department of Educational Studies, Purdue University, West Lafayette, IN 47907, USA

Funding

2024 Jiangsu Provincial Education Science Planning Project—Special Project · U.S. National Science Foundation

References (42)

Show source-provided references
1. Eye Movements in Reading and Information Processing: 20 Years of Research10.1037/0033-2909.124.3.372 ↗
2. Boonen, A.J., de Koning, B.B., Jolles, J., and van der Schoot, M. (2016). Word Problem Solving in Contemporary Math Education: A Plea for Reading Comprehension Skills Training. Front. Psychol., 7.10.3389/fpsyg.2016.00191 ↗
3. One-Step ‘Change’ and ‘Compare’ Word Problems: Focusing on Eye-Movements10.14204/ejrep.37.14133 ↗
4. Tracking Students’ Eye-Movements When Reading Learning Objects on Mobile Phones: A Discourse Analysis of Luganda Language Teacher -Trainees’ Reflective Observations10.56059/jl4d.v3i1.129 ↗
5. How Are Learning Strategies Reflected in The Eyes? Combining Results from Self-Reports and Eye-Tracking10.1111/bjep.12181 ↗
6. Memory Indexing: A Novel Method for Tracing Memory Processes in Complex Cognitive Tasks10.1037/a0028073 ↗
7. Eye-Tracking Data Quality as Affected by Ethnicity and Experimental Design10.3758/s13428-013-0343-0 ↗
8. Attention Guidance in Learning from a Complex Animation: Seeing is Understanding?10.1016/j.learninstruc.2009.02.010 ↗
9. Theoretical Frameworks, Methods, and Procedures for Conducting Phenomenological Studies
10. Eye Movements Reveal Students’ Strategies in Simple Equation Solving10.1007/s10763-014-9514-4 ↗
11. Eye-movements reveal children’s deliberative thinking and predict performance on arithmetic word problems10.1007/s10212-020-00461-w ↗
12. Wei, S., Lei, Q.L., Chen, Y.J., and Xin, Y.P. (2023). The Effects of Visual Cueing on Students with and Without Math Learning Difficulties in Online Problem Solving: Evidence from Eye Movement. Behav. Sci., 13.10.3390/bs13110927 ↗
13. Advanced Mouse-Tracking Analytic Techniques for Enhancing Psychological Science10.1177/1368430214538325 ↗
14. Recognizing Student Emotions using Brainwaves and Mouse Behavior Data10.4018/jdet.2013040101 ↗
15. Yamauchi, T. (2013, January 2–5). Mouse Trajectories and State Anxiety: Feature Selection with Random Forest. Proceedings of the 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, Geneva, Switzerland.10.1109/ACII.2013.72 ↗
16. Pimenta, A., Carneiro, D., Neves, J., and Novais, P. (2015). Improving User Privacy and the Accuracy of User Identification in Behavioral Biometrics, Springer.10.1007/978-3-319-23485-4_2 ↗
17. Web Application for Recording Learners’ Mouse Trajectories and Retrieving Their Study Logs for Data Analysis
18. Koedinger, K.R., Baker, R.S.J., Cunningham, K., Skogsholm, A., Leber, B., and Stamper, J. (2010). A Data Repository for the EDM Community: The PSLC DataShop. Handbook of Educational Data Mining, CRC Press.
19. Visualization of Eye Tracking Data: A Taxonomy and Survey10.1111/cgf.13079 ↗
20. AOI Rivers for Visualizing Dynamic Eye Gaze Frequencies10.1111/cgf.12115 ↗
21. Visual Analytics for Mobile Eye Tracking10.1109/TVCG.2016.2598695 ↗
22. FeaturEyeTrack: Automatic Matching of Eye Tracking Data with Map Features on Interactive Maps10.1007/s10707-019-00344-3 ↗
23. EyeMap: A Software System for Visualizing and Analyzing Eye Movement Data in Reading10.3758/s13428-011-0156-y ↗
24. Thakur, S., and Hanson, A.J. (2010, January 26–29). A 3D Visualization of Multiple Time Series on Maps. Proceedings of the 14th International Conference on Information Visualisation (IV), London, UK.10.1109/IV.2010.54 ↗
25. MouseTracker: Software for Studying Real-Time Mental Processing Using a Computer Mouse-Tracking Method10.3758/BRM.42.1.226 ↗
26. The Action Dynamics of Overcoming the Truth10.3758/PBR.17.4.486 ↗
27. Hierarchical Aggregation for Information Visualization: Overview, Techniques, and Design Guidelines10.1109/TVCG.2009.84 ↗
28. Krassanakis, V., and Kesidis, A.L. (2020). MatMouse: A Mouse Movements Tracking and Analysis Toolbox for Visual Search Experiments. Multimodal Technol. Interact., 4.10.3390/mti4040083 ↗
29. Decision Landscapes: Visualizing Mouse-Tracking Data10.1098/rsos.170482 ↗
30. Shneiderman, B. (1996, January 3–6). The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. Proceedings of the 1996 IEEE Symposium on Visual Languages, Boulder, CO, USA.
31. Multiscale Time Activity Data Exploration via Temporal Clustering Visualization Spreadsheet10.1109/TVCG.2008.69 ↗
32. Pondering the Concept of Abstraction in (Illustrative) Visualization10.1109/TVCG.2017.2747545 ↗
33. Xin, Y.P., Kastberg, S., and Chen, Y. (2025, April 08). Conceptual Model-Based Problem Solving: A Response to Intervention Program for Students with Learning Difficulties in Mathematics (COMPS-RtI); U.S. National Science Foundation Funded Project, Grant No. 1503451, Available online: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1503451&HistoricalAwards=false.
34. The Effect of Model-Based Problem Solving on the Performance of Students Who are Struggling in Mathematics10.1177/00224669231157032 ↗
35. Visualization of Time-Varying Weather Ensembles across Multiple Resolutions10.1109/TVCG.2016.2598869 ↗
36. Variance-Based Cluster Selection Criteria in a K Means Framework for One-Mode Dissimilarity Data10.1007/s11336-017-9561-1 ↗
37. Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection10.1145/2733381 ↗
38. Investigating the Effect of Complexity Factors in Stoichiometry Problems Using Logistic Regression and Eye Tracking10.1021/ed4004113 ↗
39. Définition numérique des courbes et surfaces
40. Strokes of Insight: User Intent Detection and Kinematic Compression of Mouse Cursor Trails10.1016/j.ipm.2016.04.005 ↗
41. The Impact of a Conceptual Model-based Intervention Program on math problem-solving performance of at-risk English learners10.1080/10573569.2019.1702909 ↗
42. Comprehension of Arithmetic Word Problems: A Comparison of Successful and Unsuccessful Problem Solvers10.1037/0022-0663.87.1.18 ↗