Voice-Controlled Robotics in Early Education: Implementing and Validating Child-Directed Interactions Using a Collaborative Robot and Artificial Intelligence
Source abstract
Abstract
This article introduces a voice-controlled robotic system for early education, enabling children as young as four to interact with robots using natural voice commands. Recognizing the challenges posed by programming languages and robot theory for young learners, this study leverages recent advancements in artificial intelligence, such as large language models, to make robots more intelligent and easier to use. This innovative approach fosters a natural and intuitive interaction between the child and the robot, effectively removing barriers to access and expanding the educational possibilities of robotics in the classroom. In this context, a software pipeline is proposed that translates voice commands into robot actions. Each component is tested using different deep learning models and cloud services to determine their suitability, with the best ones being selected. Finally, the chosen setup is validated through an integration test involving children aged 4 to 6 years. Preliminary results demonstrate the system’s capability to accurately recognize and execute voice commands, highlighting its potential as a valuable educational tool for early education.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Robotics and Automated Systems · Social Robot Interaction and HRI · AI in Service Interactions
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Facultad de Ingeniería, Arquitectura y Diseño, Universidad San Sebastián, Lago Panguipulli 1390, Puerto Montt 5501842, Chile
Facultad de Educación, Universidad San Sebastián, Lago Panguipulli 1390, Puerto Montt 5501842, Chile
Departamento de Ingeniería Eléctrica y Electrónica, Facultad de Ingeniería, Universidad del Bío-Bío, Concepción 4051381, Chile
Computer Vision Center, Edifici O, Campus UAB, 08193 Bellaterra, Spain
Computer Science Department, Universitat Autònoma de Barcelona, Campus UAB, 08193 Bellaterra, Spain
Funding
National Research and Development Agency
References (30)
Show source-provided references
1. Castro, A., Medina, J., Aguilera, C.A., Ramirez, M., and Aguilera, C. (2023). Robotics Education in STEM Units: Breaking Down Barriers in Rural Multigrade Schools. Sensors, 23.10.3390/s23010387 ↗
2. An Educational Robotics Course: Examination of Educational Potentials and Pre-service Teachers’ Experiences
3. Karagiannidis, C., Politis, P., and Karasavvidis, I. (2014). Research on e-Learning and ICT in Education: Technological, Pedagogical and Instructional Perspectives, Springer.10.1007/978-1-4614-6501-0 ↗
4. Garvis, S., and Keane, T. (2023). Technological Innovations in Education: Applications in Education and Teaching, Springer Nature.10.1007/978-981-99-2785-2 ↗
5. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4–9). Attention is All You Need. Proceedings of the Neural Information Processing Systems, Long Beach, CA, USA.
6. Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. (2023, January 23–29). Robust Speech Recognition via Large-Scale Weak Supervision. Proceedings of the International Conference on Machine Learning, Honolulu, HI, USA.
7. Lv, W., Xu, S., Zhao, Y., Wang, G., Wei, J., Cui, C., Du, Y., Dang, Q., and Liu, Y. (2023). DETRs Beat YOLOs on Real-time Object Detection. arXiv.
8. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
9. Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., and Jitsev, J. (2023, January 18–22). Reproducible scaling laws for contrastive language-image learning. Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00276 ↗
10. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18–24). Learning Transferable Visual Models From Natural Language Supervision. Proceedings of the ICML, Virtual Event.
11. PopBots: Designing an Artificial Intelligence Curriculum for Early Childhood Education
12. Construyendo conceptos sobre electricidad en infantil mediante actividades de indagación
13. Precursor models and early years science learning: A case study related to the water state changes10.1007/s10643-019-00937-5 ↗
14. Mendez, E., Ochoa, O., Olivera-Guzman, D., Soto-Herrera, V.H., Luna-Sánchez, J.A., Lucas-Dophe, C., Lugo-del Real, E., Ayala-Garcia, I.N., Alvarado Perez, M., and González, A. (2024). Integration of Deep Learning and Collaborative Robot for Assembly Tasks. Appl. Sci., 14.10.3390/app14020839 ↗
15. A Hand-Drawn Language for Human–Robot Collaboration in Wood Stereotomy10.1109/ACCESS.2023.3314337 ↗
16. (2024, January 29). Leopard-Picovoice Speech-to-Text Engine. Available online: https://picovoice.ai/docs/leopard/.
17. (2024, January 29). Vosk Speech Recognition Toolkit: Offline Speech Recognition API for Android, iOS, Raspberry Pi and Servers with Python, Java, C# and Node. Available online: https://github.com/alphacep/vosk-api.
18. (2024, January 29). Google Cloud Speech-to-Text. Available online: https://cloud.google.com/speech-to-text/.
19. Team, G., Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A.M., and Hauth, A. (2023). Gemini: A Family of Highly Capable Multimodal Models. arXiv.
20. OpenAI Introducing ChatGPT (2024, January 10). OpenAI. Available online: https://openai.com/blog/chatgpt.
21. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S.E., Fu, C., and Berg, A.C. (2016, January 11–14). SSD: Single Shot MultiBox Detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
22. Cai, Z., and Vasconcelos, N. (2018, January 18–23). Cascade R-CNN: Delving into High Quality Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00644 ↗
23. Mask R-CNN10.1109/TPAMI.2018.2844175 ↗
24. Soviany, P., and Ionescu, R.T. (2018). Optimizing the Trade-off between Single-Stage and Two-Stage Object Detectors using Image Difficulty Prediction. arXiv.10.1109/SYNASC.2018.00041 ↗
25. Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv.
26. Jocher, G., Chaurasia, A., Stoken, A., Borovec, J., Kwon, Y., Michael, K., Fang, J., Yifu, Z., Wong, C., and Montes, D. (2022). Ultralytics/yolov5: V7. 0-YOLOv5 SOTA realtime instance segmentation. Zenodo.
27. A new technique for fully autonomous and efficient 3D robotics hand/eye calibration10.1109/70.34770 ↗
28. Unity Technologies (2024, March 09). Unity Perception Package. Available online: https://github.com/Unity-Technologies/com.unity.perception.
29. Jocher, G., Chaurasia, A., and Qiu, J. (2024, March 09). Ultralytics YOLOv8. Available online: https://github.com/ultralytics/ultralytics.
30. Lin, T., Maire, M., Belongie, S.J., Bourdev, L.D., Girshick, R.B., Hays, J., Perona, P., Ramanan, D., Doll’a r, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common Objects in Context. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗