An Alternative Audio-Tactile Method of Presenting Structural Information Contained in Mathematical Drawings Adapted to the Needs of the Blind
Source abstract
Abstract
Alternative methods of presenting the information contained in mathematical images, which are adapted to the needs of blind people, are significant challenges in modern education. This article presents an alternative multimodal method that substitutes the sense of sight with the sense of touch and hearing to convey graphical information. The developed method was evaluated at a center specializing in the education of the blind in Poland, on a group of 46 students aged 15–19. They solved a set of 60 high school-level problems on geometry, mathematical analysis, and various types of graphs. We assessed the mechanisms introduced for the sense of touch and hearing, as well as the overall impression of the users. The system usability scale and the NASA task load index tests were used in the evaluation. The results obtained indicate an overall increase in user satisfaction and usefulness of the proposed approach and a reduction in the workload during exercise solving. The results also show a significant impact of the proposed navigation modes on the average time to reach objects in the drawing. Therefore, the presented method could significantly contribute to the development of systems supporting multimodal education for people with blindness.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Tactile and Sensory Interactions · Gaze Tracking and Assistive Technology · Interactive and Immersive Displays
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Distributed Systems and Informatic Devices, Silesian University of Technology, 44100 Gliwice, Poland
Department of Medical Informatics and Artificial Intelligence, Silesian University of Technology, 41800 Zabrze, Poland
Funding
program Research University Excellence Initiative
References (50)
Show source-provided references
1. Causes of Blindness and Vision Impairment in 2020 and Trends over 30 Years, and Prevalence of Avoidable Blindness in Relation to VISION 2020: The Right to Sight: An Analysis for the Global Burden of Disease Study10.1016/S2214-109X(20)30489-7 ↗
2. (2022, July 06). Blindness and Vision Impairment. Available online: https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment.
3. Computer Aided Math Learning as a Tool to Assess and Increase Motivation in Learning Math by Visually Impaired Students
4. Tutoring Math Platform Accessible for Visually Impaired People10.1016/j.compbiomed.2017.06.003 ↗
5. Tactile Graphics in School Education: Perspectives from Pupils10.1177/026461960101900204 ↗
6. Brayda, L., Leo, F., Baccelliere, C., Ferrari, E., and Vigini, C. (2018). Updated Tactile Feedback with a Pin Array Matrix Helps Blind People to Reduce Self-Location Errors. Micromachines, 9.10.3390/mi9070351 ↗
7. (2022, November 06). Guidelines and Standards for Tactile Graphics. Available online: http://www.brailleauthority.org/tg/.
8. Late- but Not Early-Onset Blindness Impairs the Development of Audio-Haptic Multisensory Integration10.1111/desc.13001 ↗
9. Development of Multisensory Integration Following Prolonged Early-Onset Visual Deprivation10.1016/j.cub.2021.08.060 ↗
10. Designing Sensory-Substitution Devices: Principles, Pitfalls and Potential1
11. Shape Perception and Navigation in Blind Adults
12. Silva, C.S., and Wimalaratne, P. (2016, January 16–19). Sensor Fusion for Visually Impaired Navigation in Constrained Spaces. Proceedings of the 2016 IEEE International Conference on Information and Automation for Sustainability (ICIAfS), Galle, Sri Lanka.10.1109/ICIAFS.2016.7946537 ↗
13. Age-Related Changes in Cutaneous Sensation in the Healthy Human Hand10.1007/s11357-012-9429-3 ↗
14. Inclusive Education: Perception of Visually Impaired Students, Students Without Disability, and Teachers in Ghana10.1177/2158244018807791 ↗
15. Ferro, T.J., and Pawluk, D.T.V. (2013, January 21–23). Automatic Image Conversion to Tactile Graphic. Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2013, Bellevue, WA, USA.10.1145/2513383.2513406 ↗
16. Tactile Graphics with a Voice10.1145/2854005 ↗
17. Götzelmann, T., and Winkler, K. (2015, January 1–3). SmartTactMaps: A Smartphone-Based Approach to Support Blind Persons in Exploring Tactile Maps. Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments, PETRA 2015, Corfu, Greece.10.1145/2769493.2769497 ↗
18. Holloway, L., Marriott, K., and Butler, M. (2018, January 21–26). Accessible Maps for the Blind: Comparing 3D Printed Models with Tactile Graphics. Proceedings of the Conference on Human Factors in Computing Systems, Montreal, QC, Canada.10.1145/3173574.3173772 ↗
19. An Empirical Approach on the Design of Tactile Maps and Diagrams: The Cognitive Tactualization Approach10.1177/0264619606063402 ↗
20. Exploration of Location-Aware You-Are-Here Maps on a Pin-Matrix Display10.1109/THMS.2015.2477999 ↗
21. Multi-Touch Interaction Data Analysis System (MIDAS) for 2-D Tactile Display Research
22. An Example of the Availability of SVG Mathematical Graphics on Touch Screens for the Blind Supporting Remote Learning
23. HCI Guidelines for Designing Website for Blinds
24. Pearl, C. (2016). Designing Voice User Interfaces: Principles of Conversational Experiences, O’Reilly Media Inc.
25. Dhaher, Y., and Clements, R. (2017). A Virtual Haptic Platform to Assist Seeing Impaired Learning: Proof of Concept Abstract. J. Blind. Innov. Res., 7.10.5241/7-123 ↗
26. Khusro, S., Shah, B., Khan, I., and Rahman, S. (2022). Haptic Feedback to Assist Blind People in Indoor Environment Using Vibration Patterns. Sensors, 22.10.3390/s22010361 ↗
27. Development of an Audio-Haptic Virtual Interface for Navigation of Large-Scale Environments for People Who Are Blind
28. Haptic Exploratory Strategies and Children Who Are Blind and Have Additional Disabilities10.1177/0145482X0409800210 ↗
29. Evaluation of Multimodal Graphs for Blind People10.1007/s10209-002-0042-6 ↗
30. Enriching Graphic Maps to Enable Multimodal Interaction by Blind People
31. Multi-Sensual Augmented Reality in Interactive Accessible Math Tutoring System for Flipped Classroom
32. Understanding Contextual Interactions to Design Navigational Context-Aware Applications
33. Multimodal User Interface System for Blind and “Visually Occupied” Users: Ergonomic Evaluation of the Haptic and Auditive Dimensions
34. Melfi, G., Müller, K., Schwarz, T., Jaworek, G., and Stiefelhagen, R. (2020, January 25–30). Understanding What You Feel: A Mobile Audio-Tactile System for Graphics Used at Schools with Students with Visual Impairment. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3313831.3376508 ↗
35. Visually Augmented Audio-Tactile Graphics for Visually Impaired People
36. Liu, H., Sun, F., Gu, J., and Deng, L. (2022). SF-YOLOv5: A Lightweight Small Object Detection Algorithm Based on Improved Feature Fusion Mode. Sensors, 22.10.3390/s22155817 ↗
37. Maćkowski, M., and Brzoza, P. (2022). Accessible Tutoring Platform Using Audio-Tactile Graphics Adapted for Visually Impaired People. Sensors, 22.10.3390/s22228753 ↗
38. Miesenberger, K., Kouroupetroglou, G., Mavrou, K., Manduchi, R., Covarrubias Rodriguez, M., and Penáz, P. Mobile E-Learning Platform for Audio-Tactile Graphics Presentation. Proceedings of the Computers Helping People with Special Needs.
39. Research on Cognitive Load Theory: Application to E-Learning10.1007/BF02504801 ↗
40. Implications of Cognitive Load Theory for Multimedia Learning10.1017/CBO9781139547369.004 ↗
41. Multimodal Presentation of Interactive Audio-Tactile Graphics Supporting the Perception of Visual Information by Blind People
42. Maćkowski, M., Kawulok, M., Brzoza, P., and Spinczyk, D. (2023). Methods and Tools Supporting the Learning and Teaching of Mathematics Dedicated to Students with Blindness. Appl. Sci., 13.10.3390/app13127240 ↗
43. Blind-Friendly User Interfaces—A Pilot Study on Improving the Accessibility of Touchscreen Interfaces10.1007/s11042-018-7094-y ↗
44. Simonnet, M., Brock, A.M., Serpa, A., Oriola, B., and Jouffrais, C. (2019). Comparing Interaction Techniques to Help Blind People Explore Maps on Small Tactile Devices. Multimodal Technol. Interact., 3.10.3390/mti3020027 ↗
45. (2023, July 15). EU Gas Consumption Decreased by 17.7%. Available online: https://ec.europa.eu/eurostat/en/web/products-eurostat-news/w/ddn-20230419-1.
46. SUS: A Quick and Dirty Usability Scale
47. Krainz, E., Moser, W., Lind, V., and Dornhofer, M. (2016, January 6–9). Accessible Way Finding on Mobile Devices for Different User Groups. Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, MobileHCI 2016, Florence, Italy.10.1145/2957265.2961847 ↗
48. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research10.1016/S0166-4115(08)62386-9 ↗
49. An Alternative Method of Audio-Tactile Presentation of Graphical Information in Mathematics Adapted to the Needs of Blind10.1016/j.ijhcs.2023.103122 ↗
50. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale