Using Chatbots as AI Conversational Partners in Language Learning
Source abstract
Abstract
Recent advances in Artificial Intelligence (AI) and machine learning have paved the way for the increasing adoption of chatbots in language learning. Research published to date has mostly focused on chatbot accuracy and chatbot–human communication from students’ or in-service teachers’ perspectives. This study aims to examine the knowledge, level of satisfaction and perceptions concerning the integration of conversational AI in language learning among future educators. In this mixed method research based on convenience sampling, 176 undergraduates from two educational settings, Spain (n = 115) and Poland (n = 61), interacted autonomously with three conversational agents (Replika, Kuki, Wysa) over a four-week period. A learning module about Artificial Intelligence and language learning was specifically designed for this research, including an ad hoc model named the Chatbot–Human Interaction Satisfaction Model (CHISM), which was used by teacher candidates to evaluate different linguistic and technological features of the three conversational agents. Quantitative and qualitative data were gathered through a pre-post-survey based on the CHISM and the TAM2 (technology acceptance) models and a template analysis (TA), and analyzed through IBM SPSS 22 and QDA Miner software. The analysis yielded positive results regarding perceptions concerning the integration of conversational agents in language learning, particularly in relation to perceived ease of use (PeU) and attitudes (AT), but the scores for behavioral intention (BI) were more moderate. The findings also unveiled some gender-related differences regarding participants’ satisfaction with chatbot design and topics of interaction.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: AI in Service Interactions · Social Robot Interaction and HRI · Organizational and Employee Performance
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Digital Language Learning (DL2) Research Group, University of Alicante, 03690 Alicante, Spain
Funding
the Instituto de Ciencias de la Educacion at the Univesity of Alicante
References (51)
Show source-provided references
1. Rediscovering the use of chatbots in education: A systematic literature review10.1002/cae.22326 ↗
2. Hwang, G.-J., and Chang, C.-Y. (2021). A review of opportunities and challenges of chatbots in education. Interact. Learn. Environ., 1–14.
3. Caldarini, G., Jaf, S., and McGarry, K. (2022). A literature survey of recent advances in chatbots. Information, 13.10.3390/info13010041 ↗
4. A critical review of state-of-the-art chatbot designs and applications10.1002/widm.1434 ↗
5. Chatbots: History, technology, and applications
6. Chatbots applications in education: A systematic review10.1016/j.caeai.2021.100033 ↗
7. Bots for language learning now: Current and future directions
8. Introduction to AI chatbots
9. Intelligent personal assistants: A systematic literature review10.1016/j.eswa.2020.113193 ↗
10. Allouch, M., Azaria, A., and Azoulay, R. (2021). Conversational Agents: Goals, Technologies, Vision and Challenges. Sensors, 21.10.3390/s21248448 ↗
11. A Review of the Practical Applications of Pedagogic Conversational Agents to Be Used in School and University Classrooms10.3390/digital1010002 ↗
12. Bots as language learning tools
13. Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations10.1016/j.chb.2015.02.026 ↗
14. Same benefits, different communication patterns: Comparing Children’s reading with a conversational agent vs. a human partner10.1016/j.compedu.2020.104059 ↗
15. A conversational agent to encourage willingness to communicate in the context of English as a foreign language10.1016/j.procs.2015.08.219 ↗
16. Io, H.N., and Lee, C.B. (2017, January 10–13). Chatbots and conversational agents: A bibliometric analysis. Proceedings of the 2017 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), Singapore.10.1109/IEEM.2017.8289883 ↗
17. Integrating CALL systems with chatbots as conversational partners
18. Radziwill, N.M., and Benton, M.C. (2017). Evaluating quality of chatbots and intelligent conversational agents. arXiv.
19. Artificial Intelligence (AI) chatbot as language learning medium: An inquiry10.1088/1742-6596/1387/1/012020 ↗
20. Bibauw, S., François, T., and Desmet, P. (2019). Computer Assisted Language Learning, Routledge.
21. Bibauw, S., François, T., and Desmet, P. (2022). Dialogue Systems for Language Learning: Chatbots and Beyond. The Routledge Handbook of Second Language Acquisition and Technology, Routledge.10.4324/9781351117586-12 ↗
22. Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning10.1111/jcal.12610 ↗
23. The rise of chatbots–new personal assistants in foreign language learning10.1016/j.procs.2020.02.212 ↗
24. Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners10.1016/j.chb.2017.05.045 ↗
25. Evaluating the language resources of chatbots for their potential in English as a second language10.1017/S0958344008000815 ↗
26. The linguistic accuracy of chatbots: Usability from an ESL perspective10.1515/text-2014-0018 ↗
27. Are we there yet?-A systematic literature review on chatbots in education10.3389/frai.2021.654924 ↗
28. Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions10.1016/j.chb.2018.03.051 ↗
29. Intelligent conversational agents in healthcare: Hype or hope
30. Kim, N.-Y. (2020). Chatbots and Language Learning: Effects of the Use of AI Chatbots for EFL Learning, Eliva Press.
31. Bailey, D. (2019, January 27–29). Chatbots as conversational agents in the context of language learning. Proceedings of the Fourth Industrial Revolution and Education, Dajeon, Korea.
32. Mageira, K., Pittou, D., Papasalouros, A., Kotis, K., Zangogianni, P., and Daradoumis, A. (2022). Educational AI Chatbots for Content and Language Integrated Learning. Appl. Sci., 12.10.3390/app12073239 ↗
33. Nkomo, L.M., and Alm, A. (2022). Sentiment Analysis: Capturing Chatbot Experiences of Informal Language Learners. Emerging Concepts in Technology-Enhanced Language Teaching and Learning, IGI Global.10.4018/978-1-7998-8981-6.ch011 ↗
34. Chatbot learning partners: Connecting learning experiences, interest and competence10.1016/j.chb.2018.12.023 ↗
35. Jeon, J. (2021). Exploring AI chatbot affordances in the EFL classroom: Young learners’ experiences and perspectives. Comput. Assist. Lang. Learn., 1–26.10.1080/09588221.2021.2021241 ↗
36. Teachers’ Views on the Use of Chatbots to Support English Language Teaching in a Mobile Environment10.3991/ijet.v16i20.24917 ↗
37. Chocarro, R., Cortiñas, M., and Marcos-Matás, G. (2021). Teachers’ attitudes towards chatbots in education: A technology acceptance model approach considering the effect of social language, bot proactiveness, and users’ characteristics. Educ. Stud., 1–19.10.1080/03055698.2020.1850426 ↗
38. A chatbot for learning Chinese: Learning achievement and technology acceptance10.1177/0735633120929622 ↗
39. Pre-service primary English teachers’ AI chatbots10.30961/lr.2020.56.1.97 ↗
40. Perceptions of preservice teachers on AI chatbots in English education
41. Pardede, P. (2019). Mixed methods research designs in EFL. PROCEEDING English Education Department Collegiate Forum (EED CF) 2015–2018, UKI Press.
42. Davis, F.D., Marangunic, A., and Granic, A. (2020). Technology Acceptance Model: 30 Years Of Tam, Springer.
43. A theoretical extension of the technology acceptance model: Four longitudinal field studies10.1287/mnsc.46.2.186.11926 ↗
44. Doing template analysis
45. Li, Y., Chen, C.-Y., Yu, D., Davidson, S., Hou, R., Yuan, X., Tan, Y., Pham, D., and Yu, Z. (2022, January 1–3). Using Chatbots to Teach Languages. Proceedings of the Ninth ACM Conference on Learning at Scale, New York, NY, USA.10.1145/3491140.3528329 ↗
46. Kılıçkaya, F. (2020). Using a chatbot, Replika, to practice writing through conversations in L2 English: A Case study. New Technological Applications for Foreign and Second Language Learning and Teaching, IGI Global.10.4018/978-1-7998-2591-3.ch011 ↗
47. User Feedback Analysis of an AI-Enabled CBT Mental Health Application (Wysa)10.2196/35668 ↗
48. An empathy-driven, conversational artificial intelligence agent (Wysa) for digital mental well-being: Real-world data evaluation mixed-methods study10.2196/12106 ↗
49. Feine, J., Gnewuch, U., Morana, S., and Maedche, A. (2019). Gender bias in chatbot design. Lecture Notes in Computer Science, Proceedings of the International Workshop on Chatbot Research and Design, Amsterdam, The Netherlands, 19–20 November 2019, Springer.10.1007/978-3-030-39540-7_6 ↗
50. Maedche, A. (2019, January 19–20). Gender bias in chatbot design. Proceedings of the Third International Workshop, Conversations 2019, Amsterdam, The Netherlands.
51. Farah, J.C., Sharma, V., Ingram, S., and Gillet, D. (2021, January 9–11). Conveying the Perception of Humor Arising from Ambiguous Grammatical Constructs in Human-Chatbot Interaction. Proceedings of the 9th International Conference on Human-Agent Interaction, online.10.1145/3472307.3484677 ↗