A Survey on Recent Advances in AI and Vision-Based Methods for Helping and Guiding Visually Impaired People
Source abstract
Abstract
We present in this paper the state of the art and an analysis of recent research work and achievements performed in the domain of AI-based and vision-based systems for helping blind and visually impaired people (BVIP). We start by highlighting the recent and tremendous importance that AI has acquired following the use of convolutional neural networks (CNN) and their ability to solve image classification tasks efficiently. After that, we also note that VIP have high expectations about AI-based systems as a possible way to ease the perception of their environment and to improve their everyday life. Then, we set the scope of our survey: we concentrate our investigations on the use of CNN or related methods in a vision-based system for helping BVIP. We analyze the existing surveys, and we study the current work (a selection of 30 case studies) using several dimensions such as acquired data, learned models, and human–computer interfaces. We compare the different approaches, and conclude by analyzing future trends in this domain.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Tactile and Sensory Interactions · Video Surveillance and Tracking Methods · Retinal Imaging and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
LIFAT, University of Tours, FR-37000 Tours, France
CETU ILIAD3, University of Tours, FR-37000 Tours, France
References (55)
Show source-provided references
1. Elmannai, W., and Elleithy, K. (2017). Sensor-based assistive devices for visually-impaired people: Current status, challenges, and future directions. Sensors, 17.10.3390/s17030565 ↗
2. Wearable assistive devices for visually impaired: A state of the art survey10.1016/j.patrec.2018.10.031 ↗
3. Techniques for Constructing Indoor Navigation Systems for the Visually Impaired: A Review10.1109/THMS.2020.3016051 ↗
4. An insight into assistive technology for the visually impaired and blind people: State-of-the-art and future trends10.1007/s12193-016-0235-6 ↗
5. Analysis of Navigation Assistants for Blind and Visually Impaired People: A Systematic Review10.1109/ACCESS.2021.3052415 ↗
6. Object detection with deep learning: A review10.1109/TNNLS.2018.2876865 ↗
7. A survey of deep learning-based object detection10.1109/ACCESS.2019.2939201 ↗
8. Ignatov, A., Timofte, R., Chou, W., Wang, K., Wu, M., Hartley, T., and Van Gool, L. (2018, January 8–14). AI benchmark: Running deep neural networks on android smartphones. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany.10.1007/978-3-030-11021-5_19 ↗
9. Leo, M., Furnari, A., Medioni, G.G., Trivedi, M., and Farinella, G.M. (2018, January 8–14). Deep learning for assistive computer vision. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany.10.1007/978-3-030-11024-6_1 ↗
10. El-Zahraa El-Taher, F., Taha, A., Courtney, J., and Mckeever, S. (2021). A systematic review of urban navigation systems for visually impaired people. Sensors, 21.10.3390/s21093103 ↗
11. Developing walking assistants for visually impaired people: A review10.1109/JSEN.2018.2890423 ↗
12. Real, S., and Araujo, A. (2019). Navigation systems for the blind and visually impaired: Past work, challenges, and open problems. Sensors, 19.10.3390/s19153404 ↗
13. Plikynas, D., Žvironas, A., Budrionis, A., and Gudauskis, M. (2020). Indoor navigation systems for visually impaired persons: Mapping the features of existing technologies to user needs. Sensors, 20.10.3390/s20030636 ↗
14. Kuriakose, B., Shrestha, R., and Sandnes, F.E. (2020). Tools and Technologies for Blind and Visually Impaired Navigation Support: A Review. IETE Tech. Rev., 1–16.10.1080/02564602.2020.1819893 ↗
15. A spiking neural network model for obstacle avoidance in simulated prosthetic vision10.1016/j.ins.2017.03.006 ↗
16. NavCog3 in the wild: Large-scale blind indoor navigation assistant with semantic features10.1145/3340319 ↗
17. Murata, M., Ahmetovic, D., Sato, D., Takagi, H., Kitani, K.M., and Asakawa, C. (2018, January 21–25). Smartphone-based indoor localization for blind navigation across building complexes. Proceedings of the 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom), Pisa, Italy.10.1109/PERCOM.2018.8444593 ↗
18. Sato, D., Oh, U., Naito, K., Takagi, H., Kitani, K., and Asakawa, C. (November, January 29). Navcog3: An evaluation of a smartphone-based blind indoor navigation assistant with semantic features in a large-scale environment. Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, Baltimore, MD, USA.
19. Virtual-blind-road following-based wearable navigation device for blind people10.1109/TCE.2018.2812498 ↗
20. Smart guiding glasses for visually impaired people in indoor environment10.1109/TCE.2017.014980 ↗
21. Chen, Z., Liu, X., Kojima, M., Huang, Q., and Arai, T. (2021). A Wearable Navigation Device for Visually Impaired People Based on the Real-Time Semantic Visual SLAM System. Sensors, 21.10.3390/s21041536 ↗
22. Wang, H.C., Katzschmann, R.K., Teng, S., Araki, B., Giarré, L., and Rus, D. (June, January 29). Enabling independent navigation for visually impaired people through a wearable vision-based feedback system. Proceedings of the 2017 IEEE International Conference on robotics and Automation (ICRA), Singapore.10.1109/ICRA.2017.7989772 ↗
23. Malek, S., Melgani, F., Mekhalfi, M.L., and Bazi, Y. (2017). Real-time indoor scene description for the visually impaired using autoencoder fusion strategies with visible cameras. Sensors, 17.10.3390/s17112641 ↗
24. Lin, S., Cheng, R., Wang, K., and Yang, K. (2018). Visual localizer: Outdoor localization based on convnet descriptor and global optimization for visually impaired pedestrians. Sensors, 18.10.3390/s18082476 ↗
25. Yang, K., Wang, K., Bergasa, L.M., Romera, E., Hu, W., Sun, D., Sun, J., Cheng, R., Chen, T., and López, E. (2018). Unifying terrain awareness for the visually impaired through real-time semantic segmentation. Sensors, 18.10.3390/s18051506 ↗
26. Simões, W.C., Silva, Y.M., Pio, J.L.d.S., Jazdi, N., and F de Lucena, V. (2020). Audio Guide for Visually Impaired People Based on Combination of Stereo Vision and Musical Tones. Sensors, 20.10.3390/s20010151 ↗
27. Hu, W., Wang, K., Yang, K., Cheng, R., Ye, Y., Sun, L., and Xu, Z. (2020). A comparative study in real-time scene sonification for visually impaired people. Sensors, 20.10.3390/s20113222 ↗
28. Son, H., Krishnagiri, D., Jeganathan, V.S., and Weiland, J. (2020, January 20–24). Crosswalk guidance system for the blind. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine &Biology Society (EMBC), Montreal, QC, Canada.10.1109/EMBC44109.2020.9176623 ↗
29. Bai, J., Liu, Z., Lin, Y., Li, Y., Lian, S., and Liu, D. (2019). Wearable travel aid for environment perception and navigation of visually impaired people. Electronics, 8.10.3390/electronics8060697 ↗
30. Lin, Y., Wang, K., Yi, W., and Lian, S. (2019, January 27–28). Deep learning based wearable assistive system for visually impaired people. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Korea.10.1109/ICCVW.2019.00312 ↗
31. Dimas, G., Diamantis, D.E., Kalozoumis, P., and Iakovidis, D.K. (2020). Uncertainty-Aware Visual Perception System for Outdoor Navigation of the Visually Challenged. Sensors, 20.10.3390/s20082385 ↗
32. An indoor wayfinding system based on geometric features aided graph SLAM for the visually impaired10.1109/TNSRE.2017.2682265 ↗
33. Vision-based mobile indoor assistive navigation aid for blind people10.1109/TMC.2018.2842751 ↗
34. Mahida, P., Shahrestani, S., and Cheung, H. (2020). Deep Learning-Based Positioning of Visually Impaired People in Indoor Environments. Sensors, 20.10.3390/s20216238 ↗
35. Sight-to-Sound Human-Machine Interface for Guiding and Navigating Visually Impaired People10.1109/ACCESS.2020.3029426 ↗
36. Lin, B.S., Lee, C.C., and Chiang, P.Y. (2017). Simple smartphone-based guiding system for visually impaired people. Sensors, 17.10.3390/s17061371 ↗
37. Neugebauer, A., Rifai, K., Getzlaff, M., and Wahl, S. (2020). Navigation aid for blind persons by visual-to-auditory sensory substitution: A pilot study. PLoS ONE, 15.10.1371/journal.pone.0237344 ↗
38. Tapu, R., Mocanu, B., and Zaharia, T. (2017). DEEP-SEE: Joint object detection, tracking and recognition with application to visually impaired navigational assistance. Sensors, 17.10.3390/s17112473 ↗
39. Deep-see face: A mobile face recognition system dedicated to visually impaired people10.1109/ACCESS.2018.2870334 ↗
40. Kacorri, H., Kitani, K.M., Bigham, J.P., and Asakawa, C. (2017, January 6–11). People with visual impairment training personal object recognizers: Feasibility and challenges. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.10.1145/3025453.3025899 ↗
41. Ahmetovic, D., Sato, D., Oh, U., Ishihara, T., Kitani, K., and Asakawa, C. (2020, January 25–30). Recog: Supporting blind people in recognizing personal objects. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3313831.3376143 ↗
42. Lo Valvo, A., Croce, D., Garlisi, D., Giuliano, F., Giarré, L., and Tinnirello, I. (2021). A Navigation and Augmented Reality System for Visually Impaired People. Sensors, 21.10.3390/s21093061 ↗
43. Joshi, R.C., Yadav, S., Dutta, M.K., and Travieso-Gonzalez, C.M. (2020). Efficient Multi-Object Detection and Smart Navigation Using Artificial Intelligence for Visually Impaired People. Entropy, 22.10.3390/e22090941 ↗
44. Grayson, M., Thieme, A., Marques, R., Massiceti, D., Cutrell, E., and Morrison, C. (2020, January 25–30). A dynamic AI system for extending the capabilities of blind people. Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3334480.3383142 ↗
45. Deep learning-based obstacle detection and classification with portable uncalibrated patterned light10.1109/JSEN.2018.2865306 ↗
46. Enhancing perception for the visually impaired with deep learning techniques and low-cost wearable sensors10.1016/j.patrec.2019.03.008 ↗
47. A highly accurate and reliable data fusion framework for guiding the visually impaired10.1109/ACCESS.2018.2817164 ↗
48. Wang, L., Famouri, M., and Wong, A. (2020). DepthNet Nano: A Highly Compact Self-Normalizing Neural Network for Monocular Depth Estimation. arXiv.
49. Deep Model Compression and Architecture Optimization for Embedded Systems: A Survey10.1007/s11265-020-01596-1 ↗
50. Luo, Y., Yin, L., Bai, W., and Mao, K. (2020). An Appraisal of Incremental Learning Methods. Entropy, 22.10.3390/e22111190 ↗
51. CNN-based encoder-decoder networks for salient object detection: A comprehensive review and recent advances10.1016/j.ins.2020.09.003 ↗
52. Seng, K.P., Lee, P.J., and Ang, L.M. (2021). Embedded Intelligence on FPGA: Survey, Applications and Challenges. Electronics, 10.10.3390/electronics10080895 ↗
53. Accelerating CNN Inference on ASICs: A Survey10.1016/j.sysarc.2020.101887 ↗
54. Ang, L.M., and Seng, K.P. (2021). GPU-Based Embedded Intelligence Architectures and Applications. Electronics, 10.10.3390/electronics10080952 ↗
55. Chang, I., Castillo, J., and Montes, H. (2022). Technology-Based Social Innovation: Smart City Inclusive System for Hearing Impairment and Visual Disability Citizens. Sensors, 22.10.3390/s22030848 ↗