journal article·2025-12-24Open access

An Intelligent English-Speaking Training System Using Generative AI and Speech Recognition

Ching-Ta Lu, Yen-Ju Chen, Tai-Ying Wu, Yen-Yu Lu

Applied Sciences· Volume 16· Issue 1

Abstract

English is the first foreign language most Taiwanese have encountered, yet few have achieved proficient speaking skills. This paper presents a generative AI-based English speaking training system designed to enhance oral proficiency through interactive AI agents. The system employs ChatGPT version 5.2 to generate diverse and tailored conversational scenarios, enabling learners to practice in contextually relevant situations. Spoken responses are captured via speech recognition and analyzed by a large language model, which provides intelligent scoring and personalized feedback to guide improvement. Learners can automatically generate scenario-based scripts according to their learning needs. The D-ID AI system then produces a virtual character of the AI agent, whose lip movements are synchronized with the conversation, thereby creating realistic video interactions. Learning with an AI agent, the system maintains controlled emotional expression, reduces communication anxiety, and helps learners adapt to non-native interaction, fostering more natural and confident speech production. Accordingly, the proposed system supports compelling, immersive, and personalized language learning. The experimental results indicate that repeated practice with the proposed system substantially improves English speaking proficiency.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: AI in Service Interactions · Emotion and Mood Recognition · Speech and dialogue systems

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Communications Engineering, Feng Chia University, Taichung City 407102, Taiwan

Department of Computer Science, National Chengchi University, Taipei City 116011, Taiwan

Funding

National Science and Technology Council, Taiwan

References (36)

Show source-provided references
1. Garzón, J., Lampropoulos, G., and Burgos, D. (2023). Effects of mobile learning in English language learning: A meta-analysis and research synthesis. Electronics, 12.10.3390/electronics12071595 ↗
2. Self-regulated learning method through smartphone assistance in promoting speaking ability10.17507/jltr.1304.10 ↗
3. Human-machine collaboration in language education in the age of artificial intelligence10.1177/00336882241266629 ↗
4. Empowering communities through lifelong learning: A case study of university initiatives for social engagement and personal development
5. Implementation of an AI English-speaking interactive training system using multi-model neural networks10.1109/ACCESS.2025.3592632 ↗
6. D-ID (2025, December 20). D-ID Official Website. Available online: https://www.d-id.com/.
7. English speaking practice with conversational AI: Lower secondary students’ educational experiences over time10.1016/j.caeai.2023.100164 ↗
8. Kobashikawa, S., Odakura, A., Nakamura, T., Mori, T., Endo, K., Moriya, T., Masumura, R., Aono, Y., and Minematsu, N. (2019, January 20–21). Does speaking training application with speech recognition motivate junior high school students in actual classroom?—A case study. Proceedings of the 8th ISCA Workshop on Speech and Language Technology in Education (SLaTE), Graz, Austria. Available online: https://www.isca-archive.org/slate_2019/kobashikawa19_slate.html.10.21437/SLaTE.2019-23 ↗
9. AI and English language learning outcomes
10. Benchmarking in TESOL: A study of the Malaysia education blueprint 201310.5539/elt.v7n8p23 ↗
11. Enhancing English language learning through artificial intelligence: Opportunities, challenges and the future10.54259/diajar.v4i1.3344 ↗
12. The use of technology based on artificial intelligence in English teaching and learning10.24235/eltecho.v6i2.9299 ↗
13. Wei, L. (2023). Artificial intelligence in language instruction: Impact on English learning achievement, L2 motivation, and self-regulated learning. Front. Psychol., 14.10.3389/fpsyg.2023.1261955 ↗
14. Impact of ChatGPT on learning motivation: Teachers and students’ voices10.56540/jesaf.v2i1.51 ↗
15. The role of artificial intelligence in developing English language learners’ communication skills
16. The correlation between students’ self-esteem and their English proficiency test result10.33365/jeltl.v3i2.2211 ↗
17. Teaching English in the age of AI: Embracing ChatGPT to optimize EFL materials and assessment
18. Exploring the implications of ChatGPT for language learning in higher education
19. Exploring the application of artificial intelligence in foreign language teaching: Challenges and future development10.1051/shsconf/202316803025 ↗
20. Artificial intelligence technologies and applications for language learning and teaching10.1515/jccall-2023-0015 ↗
21. Qiao, H., and Zhao, A. (2023). Artificial intelligence-based language learning: Illuminating the impact on speaking skills and self-regulation in Chinese EFL context. Front. Psychol., 14.10.3389/fpsyg.2023.1255594 ↗
22. The role of artificial intelligence technology on English language learning: A literature review
23. Artificial intelligence in education: AIEd for personalised learning pathways10.34190/ejel.20.5.2597 ↗
24. Generative artificial intelligence in education: From deceptive to disruptive
25. Investigating OpenAI’s ChatGPT potentials in generating chatbot’s dialogue for English as a foreign language learning
26. Ethical principles for artificial intelligence in education10.1007/s10639-022-11316-w ↗
27. Artificial intelligence in education: The three paradigms10.1016/j.caeai.2021.100020 ↗
28. Education in the era of generative artificial intelligence: Understanding the potential benefits of ChatGPT in promoting teaching and learning10.61969/jai.1337500 ↗
29. Opportunities and challenges of integrating generative artificial intelligence in education10.61707/8y29gv34 ↗
30. Generative artificial intelligence in higher education: Exploring ways of harnessing pedagogical practices with the assistance of ChatGPT10.47852/bonviewIJCE42022489 ↗
31. Comparing lecturers’ and students’ attitude towards the role of generative artificial intelligence systems in foreign language teaching and learning10.48161/qaj.v5n3a1583 ↗
32. Exploring the role of generative AI in enhancing language learning: Opportunities and challenges10.47852/bonviewIJCE42022495 ↗
33. The implementation of artificial intelligence in teaching foreign languages
34. Application and potential problems of generative artificial intelligence in foreign language teaching
35. Variations of box plots10.1080/00031305.1978.10479236 ↗
36. García-Varela, F., Nussbaum, M., Mendoza, M., Martínez-Troncoso, C., and Bekerman, Z. (2025). ChatGPT as a stable and fair tool for automated essay scoring. Educ. Sci., 15.10.3390/educsci15080946 ↗