An Efficient AdaBoost Algorithm with the Multiple Thresholds Classification
Source abstract
Abstract
Adaptive boost (AdaBoost) is a prominent example of an ensemble learning algorithm that combines weak classifiers into strong classifiers through weighted majority voting rules. AdaBoost’s weak classifier, with threshold classification, tries to find the best threshold in one of the data dimensions, dividing the data into two categories-1 and 1. However, in some cases, this Weak Learning algorithm is not accurate enough, showing poor generalization performance and a tendency to over-fit. To solve these challenges, we first propose a new Weak Learning algorithm that classifies examples based on multiple thresholds, rather than only one, to improve its accuracy. Second, in this paper, we make changes to the weight allocation scheme of the Weak Learning algorithm based on the AdaBoost algorithm to use potential values of other dimensions in the classification process, while the theoretical identification is provided to show its generality. Finally, comparative experiments between the two algorithms on 18 datasets on UCI show that our improved AdaBoost algorithm has a better generalization effect in the test set during the training iteration.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Imbalanced Data Classification Techniques · Face and Expression Recognition · Machine Learning and ELM
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Maritime College, Guangdong Ocean University, Zhanjiang 524091, China
College of Mathematics and Computer, Guangdong Ocean University, Zhanjiang 524091, China
Funding
Program for Scientific Research Start-up Funds of Guangdong Ocean University
References (30)
Show source-provided references
1. Dietterich, T.G. (2000). Ensemble Methods in Machine Learning. Multiple Classifier Systems, Springer.10.1007/3-540-45014-9_1 ↗
2. Ensemble machine learning paradigms in hydrology: A review10.1016/j.jhydrol.2021.126266 ↗
3. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting10.1006/jcss.1997.1504 ↗
4. Comparing the Fault Diagnosis Performances of Single Neural Networks and Two Ensemble Neural Networks Based on the Boosting Methods
5. Boosting the margin: A new explanation for the effectiveness of voting methods
6. Schapire, R.E. (2003). The Boosting Approach to Machine Learning: An Overview. Nonlinear Estimation and Classification, Springer.10.1007/978-0-387-21579-2_9 ↗
7. Improved Boosting Algorithms Using Confidence-rated Predictions10.1023/A:1007614523901 ↗
8. An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants10.1023/A:1007515423169 ↗
9. Random Forests
10. McDonald, R.A., Hand, D.J., and Eckley, I.A. (2003). An Empirical Comparison of Three Boosting Algorithms on Real Data Sets with Artificial Class Noise. Multiple Classifier Systems, Springer.10.1007/3-540-44938-8_4 ↗
11. Popular Ensemble Methods: An Empirical Study
12. Hierarchical Skin-AdaBoost-Neural Network (H-SKANN) for multi-face detection10.1016/j.asoc.2018.03.030 ↗
13. Using AdaBoost classifiers in a hierarchical framework for classifying surface images of marble slabs10.1016/j.eswa.2010.06.019 ↗
14. M-AdaBoost-A based ensemble system for network intrusion detection10.1016/j.eswa.2020.113864 ↗
15. Recognition of diseased Pinus trees in UAV images using deep learning and AdaBoost classifier10.1016/j.biosystemseng.2020.03.021 ↗
16. AdaBoost-based transfer learning method for positive and unlabelled learning problem10.1016/j.knosys.2022.108162 ↗
17. An empowered AdaBoost algorithm implementation: A COVID-19 dataset study10.1016/j.cie.2021.107912 ↗
18. AdaBoost-CNN: An adaptive boosting algorithm for convolutional neural networks to classify multi-class imbalanced datasets using transfer learning10.1016/j.neucom.2020.03.064 ↗
19. MF-Adaboost: LDoS attack detection based on multi-features and improved Adaboost10.1016/j.future.2019.12.034 ↗
20. The improved AdaBoost algorithms for imbalanced data classification10.1016/j.ins.2021.03.042 ↗
21. BPSO-Adaboost-KNN ensemble learning algorithm for multi-class imbalanced data classification10.1016/j.engappai.2015.09.011 ↗
22. Milling chatter detection by multi-feature fusion and Adaboost-SVM10.1016/j.ymssp.2021.107671 ↗
23. Forests of nested dichotomies10.1016/j.patrec.2009.09.015 ↗
24. Robust boosting classification models with local sets of probability distributions10.1016/j.knosys.2014.02.007 ↗
25. Additive logistic regression: A statistical view of boosting (With discussion and a rejoinder by the authors)10.1214/aos/1016218223 ↗
26. Trevor, H., Robert, T., and Jerome, H. (2001). The Elements of Sta-tistical Learning, Springer.
27. Functional gradient ascent for Probit regression10.1016/j.patcog.2012.06.006 ↗
28. Bylander, T., and Tate, L. (2006, January 11–13). Using Validation Sets to Avoid Overfitting in AdaBoost. Proceedings of the FLAIRS 2006—Proceedings of the Nineteenth International Florida Artificial Intelligence Research Society Conference, Melbourne Beach, FL, USA.
29. Meddouri, N., Khoufi, H., and Maddouri, M.S. (2012). Diversity Analysis on Boosting Nominal Concepts. Advances in Knowledge Discovery and Data Mining, Springer.10.1007/978-3-642-30217-6_26 ↗
30. Adaboost Ensemble Data Classification Based on Diversity of Classifiers