journal article·2024Open access

Enhancing ChatGPT’s Querying Capability with Voice-Based Interaction and CNN-Based Impair Vision Detection Model

Awais Ahmad, Sohail Jabbar, Sheeraz Akram, Anand Paul, Umar Raza, Nuha Mohammed Alshuqayran

Computers, Materials & Continua· Volume 78· Issue 3

Abstract

This paper presents an innovative approach to enhance the querying capability of ChatGPT, a conversational artificial intelligence model, by incorporating voice-based interaction and a convolutional neural network (CNN)-based impaired vision detection model. The proposed system aims to improve user experience and accessibility by allowing users to interact with ChatGPT using voice commands. Additionally, a CNN-based model is employed to detect impairments in user vision, enabling the system to adapt its responses and provide appropriate assistance. This research tackles head-on the challenges of user experience and inclusivity in artificial intelligence (AI). It underscores our commitment to overcoming these obstacles, making ChatGPT more accessible and valuable for a broader audience. The integration of voice-based interaction and impaired vision detection represents a novel approach to conversational AI. Notably, this innovation transcends novelty; it carries the potential to profoundly impact the lives of users, particularly those with visual impairments. The modular approach to system design ensures adaptability and scalability, critical for the practical implementation of these advancements. Crucially, the solution places the user at its core. Customizing responses for those with visual impairments demonstrates AI’s potential to not only understand but also accommodate individual needs and preferences.
Source: openalex_inverted_index · Quality: baseline

Research context

OpenAlex topics: Artificial Intelligence in Healthcare and Education · COVID-19 diagnosis using AI · AI in Service Interactions

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

References (28)

Show source-provided references
1. Global estimates of visual impairment: 201010.1136/bjophthalmol-2011-300539 ↗
2. Explainable artificial intelligence approach in combating real-time surveillance of COVID19 pandemic from CT scan and X-ray images using ensemble model10.1007/s11227-022-04631-z ↗
3. Kashif · Innovation in Health Informatics · 2020 · 14510.1016/B978-0-12-819043-2.00006-X ↗
4. An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges10.1016/j.tbench.2023.100089 ↗
5. The human side of human-chatbot interaction: A systematic literature review of ten years of research on text-based chatbots10.1016/j.ijhcs.2021.102630 ↗
6. Student query trend assessment with semantical annotation and artificial intelligent multi-agents10.12973/eurasia.2017.00763a ↗
7. New horizons: The potential role of Openai’s ChatGPT in clinical radiology10.1016/j.jacr.2023.02.025 ↗
8. Using AI-generated suggestions from ChatGPT to optimize clinical decision support10.1093/jamia/ocad072 ↗
9. ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns10.3390/healthcare11060887 ↗
10. Conversation-based adaptive relational translation method for next POI recommendation with uncertain check-ins10.1109/TNNLS.2022.3146443 ↗
11. Malware detection in Industrial Internet of Things based on hybrid image visualization and deep learning model10.1016/j.adhoc.2020.102154 ↗
12. Plagiarism detection in students’ programming assignments based on semantics: Multimedia E-learning based smart assessment methodology10.1007/s11042-018-5827-6 ↗
13. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope
14. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models10.1371/journal.pdig.0000198 ↗
15. Revisiting the boundary between ASR and NLU in the age of conversational dialog systems10.1162/coli_a_00430 ↗
16. Digital interaction literacy model-conceptualizing competencies for literate interactions with voice-based AI systems
17. X. Qu, H. Liu, Z. Sun, X. Yin, Y. S. Ong et al., “Towards building voice-based conversational recommender systems: Datasets, potential solutions, and prospects,” arXiv preprint arXiv:2306.08219, 2023.10.1145/3539618.3591876 ↗
18. A review of chatGPT AI’s impact on several business sectors
19. V. Basmov, Y. Goldberg and R. Tsarfaty, “ChatGPT and simple linguistic inferences: Blind spots and blinds,” arXiv preprint arXiv:2305.14785, 2023.
20. A. Kuzdeuov, S. Nurgaliyev and H. A. Varol, “ChatGPT for visually impaired and blind,” 2023. [Online]. Available: https://www.techrxiv.org/articles/preprint/ChatGPT_for_Visually_Impaired_and_Blind/22047080 (accessed on 03/05/2023).10.36227/techrxiv.22047080.v1 ↗
21. J. Witte Zimmerman, D. Hudon, K. Cramer, J. St Onge, M. Fudolig et al., “A blind spot for large language models: Supradiegetic linguistic information,” arXiv preprint arXiv:2306.06794, 2023.
22. Artificial intelligence bot ChatGPT in medical research: The potential game changer as a double-edged sword10.1007/s00167-023-07355-6 ↗
23. Automated detection of mild and multi-class diabetic eye diseases using deep learning10.1007/s13755-020-00125-5 ↗
24. I. S. Gabashvili, “The impact and applications of ChatGPT: A systematic review of literature reviews,” arXiv preprint arXiv:2305.18086, 2023.
25. Early applications of ChatGPT in medical practice, education and research10.7861/clinmed.2023-0078 ↗
26. Voice assistant integrated with ChatGPT10.33022/ijcs.v12i1.3146 ↗
27. Attention-based 1D CNN-BILSTM hybrid model enhanced with fasttext word embedding for Korean voice phishing detection10.3390/math11143217 ↗
28. SIRI talks at you—An empirical investigation of voice-activated personal assistant (VAPA) usage by individuals who are blind