journal article·2022-02-11Open access

Week-Wise Student Performance Early Prediction in Virtual Learning Environment Using a Deep Explainable Artificial Intelligence

Hsing-Chung Chen, Eko Prasetyo, Shian-Shyong Tseng, Karisma Trinanda Putra, Prayitno, Sri Suning Kusumawardani, Chien-Erh Weng

Applied Sciences· Volume 12· Issue 4

Abstract

Early prediction of students’ learning performance and analysis of student behavior in a virtual learning environment (VLE) are crucial to minimize the high failure rate in online courses during the COVID-19 pandemic. Nevertheless, traditional machine learning models fail to predict student performance in the early weeks due to the lack of students’ activities’ data in a week-wise timely manner (i.e., spatiotemporal feature issues). Furthermore, the imbalanced data distribution in the VLE impacts the prediction model performance. Thus, there are severe challenges in handling spatiotemporal features, imbalanced data sets, and a lack of explainability for enhancing the confidence of the prediction system. Therefore, an intelligent framework for explainable student performance prediction (ESPP) is proposed in this study in order to provide the interpretability of the prediction results. First, this framework utilized a time-series weekly student activity data set and dealt with the VLE imbalanced data distribution using a hybrid data sampling method. Then, a combination of convolutional neural network (CNN) and long short-term memory (LSTM) was employed to extract the spatiotemporal features and develop the early prediction deep learning (DL) model. Finally, the DL model was explained by visualizing and analyzing typical predictions, students’ activities’ maps, and feature importance. The numerical results of cross-validation showed that the proposed new DL model (i.e., the combined CNN-LSTM and ConvLSTM), in the early prediction cases, performed better than the baseline models of LSTM, support vector machine (SVM), and logistic regression (LR) models.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Online Learning and Analytics · Advanced Technologies in Various Fields · Artificial Intelligence in Healthcare

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Computer Science and Information Engineering, Asia University, Taichung City 413, Taiwan

Department of Medical Research, China Medical University Hospital, China Medical University, Taichung City 404, Taiwan

Department of Information Technology, Universitas Muhammadiyah Yogyakarta, Bantul 55183, Indonesia

Department of M-Commerce and Multimedia Applications, Asia University, Taichung City 413, Taiwan

Department of Electrical Engineering, Universitas Muhammadiyah Yogyakarta, Bantul 55183, Indonesia

Department of Electrical Engineering, Politeknik Negeri Semarang, Semarang 50275, Indonesia

Department of Electrical and Information Engineering, Universitas Gadjah Mada, Yogyakarta 55281, Indonesia

Department of Telecommunication Engineering, National Kaohsiung University of Science and Technology, Kaohsiung 811, Taiwan

References (39)

Show source-provided references
1. COVID-19: 20 countries’ higher education intra-period digital pedagogy responses
2. COVID-19 and emergency eLearning: Consequences of the securitization of higher education for post-pandemic pedagogy10.1080/13523260.2020.1761749 ↗
3. Bin Mat, U., Buniyamin, N., Arsad, P.M., and Kassim, R. (2013, January 4–5). An Overview of Using Academic Analytics to Predict and Improve Students’ Achievement: A Proposed Proactive Intelligent Intervention. Proceedings of the 2013 IEEE 5th Conference on Engineering Education (ICEED), Kuala Lumpur, Malaysia.10.1109/ICEED.2013.6908316 ↗
4. Zhou, J., and Zhang, Q. (2021). A Survey Study on U.S. College Students’ Learning Experience in COVID-19. Educ. Sci., 11.10.3390/educsci11050248 ↗
5. Student and instructor perceptions of engagement after the rapid online transition of teaching due to COVID-1910.1002/nse2.20038 ↗
6. Guzsvinecz, T., and Szűcs, J. (2021). Using Analytics to Identify When Course Materials Are Accessed Relative to Online Exams during Digital Education. Educ. Sci., 11.10.3390/educsci11100576 ↗
7. Comprehensive Methods of Evaluation of Efficiency of Distance Learning System Functioning
8. Massive LMS log data analysis for the early prediction of course-agnostic student performance10.1016/j.compedu.2020.104108 ↗
9. Learning Performance Prediction with Imbalanced Virtual Learning Environment Students’ Interactions Data
10. One-step spectral rotation clustering for imbalanced high-dimensional data10.1016/j.ipm.2020.102388 ↗
11. Kraiem, M.S., Sánchez-Hernández, F., and Moreno-García, M.N. (2021). Selecting the Suitable Resampling Strategy for Imbalanced Data Classification Regarding Dataset Properties. An Approach Based on Association Models. Appl. Sci., 11.10.3390/app11188546 ↗
12. Lin, I., Loyola-González, O., Monroy, R., and Medina-Pérez, M. (2021). A Review of Fuzzy and Pattern-Based Approaches for Class Imbalance Problems. Appl. Sci., 11.10.3390/app11146310 ↗
13. Sekeroglu, B., Abiyev, R., Ilhan, A., Arslan, M., and Idoko, J.B. (2021). Systematic Literature Review on Machine Learning and Student Performance Prediction: Critical Gaps and Possible Remedies. Appl. Sci., 11.10.3390/app112210907 ↗
14. Zafari, M., Sadeghi-Niaraki, A., Choi, S.-M., and Esmaeily, A. (2021). A Practical Model for the Evaluation of High School Student Performance Based on Machine Learning. Appl. Sci., 11.10.3390/app112311534 ↗
15. Predicting Student Performance and Its Influential Factors Using Hybrid Regression and Multi-Label Classification10.1109/ACCESS.2020.3036572 ↗
16. Aljohani, N.R., Fayoumi, A., and Hassan, S.-U. (2019). Predicting At-Risk Students Using Clickstream Data in the Virtual Learning Environment. Sustainability, 11.10.3390/su11247238 ↗
17. Exploring the factors affecting MOOC retention: A survey study10.1016/j.compedu.2016.03.016 ↗
18. Wasif, M., Waheed, H., Aljohani, N.R., and Hassan, S.-U. (2019). Understanding Student Learning Behavior and Predicting Their Performance. Cognitive Computing in Technology-Enhanced Learning, IGI Global.10.4018/978-1-5225-9031-6.ch001 ↗
19. Marbouti, F., Diefes-Dux, H., and Strobel, J. (2015, January 14–17). Building Course-Specific Regression-Based Models to Identify At-Risk Students. Proceedings of the 2015 ASEE Annual Conference and Exposition Proceedings, Seattle, WA, USA. ASEE Conferences.
20. Predicting at-risk university students in a virtual learning environment via a machine learning algorithm10.1016/j.chb.2018.06.032 ↗
21. Models for early prediction of at-risk students in a course using standards-based grading10.1016/j.compedu.2016.09.005 ↗
22. Predicting Pupil’s Successfulness Factors Using Machine Learning Algorithms and Mathematical Modelling Methods
23. Student Engagement Predictions in an e-Learning System and Their Impact on Student Course Assessment Scores10.1155/2018/6347186 ↗
24. Deep learning10.1038/nature14539 ↗
25. Virtual learning environment to predict withdrawal by leveraging deep learning10.1002/int.22129 ↗
26. Okubo, F., Yamashita, T., Shimada, A., and Ogata, H. (2017, January 13). A Neural Network Approach for Students’ Performance Prediction. Proceedings of the Seventh International Learning Analytics & Knowledge Conference, Vancouver, BC, Canada.10.1145/3027385.3029479 ↗
27. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI10.1016/j.inffus.2019.12.012 ↗
28. Explaining Deep Learning using examples: Optimal feature weighting methods for twin systems using post-hoc, explanation-by-example in XAI10.1016/j.knosys.2021.107530 ↗
29. Designing a Virtual Learning Environment System for Teaching Twenty-First Century Skills to Higher Education Students in ASEAN10.1007/s10758-017-9310-7 ↗
30. Putra, K., Chen, H.-C., Ogiela, M., Chou, C.-L., Weng, C.-E., and Shae, Z.-Y. (2021). Federated Compressed Learning Edge Computing Framework with Ensuring Data Privacy for PM2.5 Prediction in Smart City Sensing Applications. Sensors, 21.10.3390/s21134586 ↗
31. Adaptive boost LS-SVM classification approach for time-series signal classification in epileptic seizure diagnosis applications10.1016/j.eswa.2020.113676 ↗
32. Long short-term memory neural network (LSTM-NN) for aquifer level time series forecasting using in-situ piezometric observations10.1016/j.jhydrol.2021.126800 ↗
33. Unbox the black-box for the medical explainable AI via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond10.1016/j.inffus.2021.07.016 ↗
34. Pulse-line intersection method with unboxed artificial intelligence for hesitant pulse wave classification10.1016/j.ipm.2021.102855 ↗
35. Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13–17). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939778 ↗
36. Feature construction and smote-based imbalance handling for multi-label learning10.1016/j.ins.2021.03.001 ↗
37. Evaluating XAI: A comparison of rule-based and example-based explanations
38. FairLens: Auditing black-box clinical decision support systems10.1016/j.ipm.2021.102657 ↗
39. Shyu, C.-R., Putra, K.T., Chen, H.-C., Tsai, Y.-Y., Hossain, K.S.M.T., Jiang, W., and Shae, Z.-Y. (2021). A Systematic Review of Federated Learning in the Healthcare Area: From the Perspective of Data Properties and Applications. Appl. Sci., 11.10.3390/app112311191 ↗