The Machine Learning-Based Dropout Early Warning System for Improving the Performance of Dropout Prediction
Source abstract
Abstract
A dropout early warning system enables schools to preemptively identify students who are at risk of dropping out of school, to promptly react to them, and eventually to help potential dropout students to continue their learning for a better future. However, the inherent class imbalance between dropout and non-dropout students could pose difficulty in building accurate predictive modeling for a dropout early warning system. The present study aimed to improve the performance of a dropout early warning system: (a) by addressing the class imbalance issue using the synthetic minority oversampling techniques (SMOTE) and the ensemble methods in machine learning; and (b) by evaluating the trained classifiers with both receiver operating characteristic (ROC) and precision–recall (PR) curves. To that end, we trained random forest, boosted decision tree, random forest with SMOTE, and boosted decision tree with SMOTE using the big data samples of the 165,715 high school students from the National Education Information System (NEIS) in South Korea. According to our ROC and PR curve analysis, boosted decision tree showed the optimal performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Online Learning and Analytics · Software Reliability and Analysis Research · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Psychology, University of Houston, Houston, TX 77004, USA
Department of Education, Ewha Womans University, Seoul 03760, Korea
References (36)
Show source-provided references
1. High School Dropouts: A Review of Issues and Evidence10.3102/00346543057002101 ↗
2. On the social costs of dropping out of school
3. Preventing Student Disengagement and Keeping Students on the Graduation Path in Urban Middle-Grades Schools: Early Identification and Effective Interventions10.1080/00461520701621079 ↗
4. How Can We Help? What We Have Learned From Recent Federal Dropout Prevention Evaluations10.1207/S15327671ESPR0701_4 ↗
5. Lamb, S., and Rice, S. (2008). Effective Strategies to Increase School Completion Report: Report to the Victorian Department of Education and Early Childhood Development.
6. Of needles and haystacks: Building an accurate statewide dropout early warning system in Wisconsin
7. Sullivan, R. (2017). Early Warning Signs. A Solution-Finding Report, Center on Innovations in Learning, Temple University.
8. Learning from class-imbalanced data: Review of methods and applications10.1016/j.eswa.2016.12.035 ↗
9. A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based Approaches10.1109/TSMCC.2011.2161285 ↗
10. Saito, T., and Rehmsmeier, M. (2015). The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets. PLoS ONE, 10.10.1371/journal.pone.0118432 ↗
11. Predicting student failure at school using genetic programming and different data mining approaches with high dimensional and imbalanced data10.1007/s10489-012-0374-8 ↗
12. Early dropout prediction using data mining: A case study with high school students10.1111/exsy.12135 ↗
13. (2018, August 01). Korean Educational Development Institute. Available online: http://cesi.kedi.re.kr/post/6662567?itemCode=03&menuId=m_02_03_03.
14. Chung, J., Kang, T., Kim, S.K., Ryoo, J.S., Lee, D., Lee, J., and Hwang, J. (2013). Policy Study on the Supporting System for Out-of-School Youth, Jeollanamdo Office of Education.
15. “Education for All?”10.1177/10442073070180020601 ↗
16. Mathis, W.J., and Trujillo, T.M. (2016). Lessons from NCLB for the Every Student Succeeds Act, National Education Policy Center.
17. Dropout early warning systems for high school students using machine learning10.1016/j.childyouth.2018.11.030 ↗
18. Chawla, N.V., Lazarevic, A., Hall, L.O., and Bowyer, K.W. (2003). SMOTEBoost: Improving Prediction of the Minority Class in Boosting. Knowledge Discovery in Databases: PKDD 2003, Springer.10.1007/978-3-540-39804-2_12 ↗
19. SMOTE: Synthetic Minority Over-sampling Technique10.1613/jair.953 ↗
20. Performance criteria for plastic card fraud detection tools10.1057/palgrave.jors.2602418 ↗
21. Chawla, N.V. (2005). Data Mining for Imbalanced Datasets: An Overview. Data Mining and Knowledge Discovery Handbook, Springer.10.1007/0-387-25465-X_40 ↗
22. Longadge, R., and Dongre, S. (2013). Class imbalance problem in data mining review. arXiv.
23. Support vector machines for classification in nonstandard situations10.1023/A:1012406528296 ↗
24. Napierała, K., Stefanowski, J., and Wilk, S. (2010). Learning from Imbalanced Data in Presence of Noisy and Borderline Examples. Rough Sets and Current Trends in Computing, Springer.10.1007/978-3-642-13529-3_18 ↗
25. Test strategies for cost-sensitive decision trees10.1109/TKDE.2006.131 ↗
26. Zhou, Z.H. (2012). Ensemble Methods, Chapman and Hall/CRC.
27. Viola, P., and Jones, M. (2001, January 8–14). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, Kauai, HI, USA.
28. Lung cancer cell identification based on artificial neural network ensembles10.1016/S0933-3657(01)00094-X ↗
29. Credit card fraud detection: A fusion approach using Dempster–Shafer theory and Bayesian learning10.1016/j.inffus.2008.04.001 ↗
30. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting10.1006/jcss.1997.1504 ↗
31. An introduction to ROC analysis10.1016/j.patrec.2005.10.010 ↗
32. Therriault, S.B., Heppen, J., O’Cummings, M., Fryer, L., and Johnson, A. (2018, August 01). Early Warning System Implementation Guide. Available online: http://www.betterhighschools.org/documents/NHSCEWSImplementationGuide.pdf.
33. Allensworth, E.M., and Easton, J.Q. (2007). What Matters for Staying On-Track and Graduating in Chicago Public High Schools: A Close Look at Course Grades, Failures, and Attendance in the Freshman Year. Research Report.
34. Tantithamthavorn, C., Hassan, A.E., and Matsumoto, K. (2018). The Impact of Class Rebalancing Techniques on the Performance and Interpretation of Defect Prediction Models. IEEE Trans. Softw. Eng., 1.
35. Cellini, S.R., and Kee, J.E. (2015). Cost-Effectiveness and Cost-Benefit Analysis. Handbook of Practical Program Evaluation, John Wiley & Sons, Inc.10.1002/9781119171386.ch24 ↗
36. Estimation and Inference of Heterogeneous Treatment Effects using Random Forests10.1080/01621459.2017.1319839 ↗