High-Dimensional, Small-Sample Product Quality Prediction Method Based on MIC-Stacking Ensemble Learning
Source abstract
Abstract
Accurate quality prediction can find and eliminate quality hazards. It is difficult to construct an accurate quality mathematical model for the production of small samples with high dimensionality due to the influence of quality characteristics and the complex mechanism of action. In addition, overfitting scenarios are prone to occur in high-dimensional, small-sample industrial product quality prediction. This paper proposes an ensemble learning and measurement model based on stacking and selects eight algorithms as the base learning model. The maximal information coefficient (MIC) is used to obtain the correlation between the base learning models. Models with low correlation and strong predictive power were chosen to build stacking ensemble models, which effectively avoids overfitting and obtains better predictive performance. To improve the prediction performance as the optimization goal, in the data preprocessing stage, boxplots, ordinary least squares (OLS), and multivariate imputation by chained equations (MICE) are used to detect and replace outliers. The CatBoost algorithm is used to construct combined features. Strong combination features were selected to construct a new feature set. Concrete slump data from the University of California Irvine (UCI) machine learning library were used to conduct comprehensive verification experiments. The experimental results show that, compared with the optimal single model, the minimum correlation stacking ensemble learning model has higher precision and stronger robustness, and a new method is provided to guarantee the accuracy of final product quality prediction.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Industrial Vision Systems and Defect Detection · Advanced Statistical Methods and Models · Fault Detection and Control Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Industrial Engineering, College of Mechanical and Electrical, Shihezi University, Shihezi 832003, China
Department of Data Science and Big Data Technology, College of Information Science and Technology, Shihezi University, Shihezi 832003, China
Funding
Science&Technology Department of Xinjiang Production and Construction Corps · Ministry of Industry&Information Technology, China
References (64)
Show source-provided references
1. A systematic literature review of machine learning methods applied to predictive maintenance10.1016/j.cie.2019.106024 ↗
2. Early warning systems for multi-variety and small batch manufacturing based on active learning10.3233/JIFS-169345 ↗
3. The Design and Performance of the Multivariate Synthetic-T Control Chart10.1080/03610920802178413 ↗
4. Experiences with industrial applications of projection methods for multivariate statistical process control10.1016/0098-1354(96)00132-9 ↗
5. A self-organizing power system stabilizer using fuzzy auto-regressive moving average (FARMA) model10.1109/60.507658 ↗
6. A multivariate exponentially weighted moving average10.2307/1269551 ↗
7. Machine learning: Trends, perspectives, and prospects10.1126/science.aaa8415 ↗
8. Data Mining and Analytics in the Process Industry: The Role of Machine Learning10.1109/ACCESS.2017.2756872 ↗
9. Jiao, A., Zhang, G., Liu, B., and Liu, W. (2020). Prediction of Manufacturing Quality of Holes Based on a BP Neural Network. Appl. Sci., 10.10.3390/app10062108 ↗
10. Particle swarm optimization10.1007/s11721-007-0002-0 ↗
11. Firefly algorithm, stochastic test functions and design optimisation10.1504/IJBIC.2010.032124 ↗
12. Proper estimation of surface roughness using hybrid intelligence based on artificial neural network and genetic algorithm10.1016/j.jmapro.2021.08.062 ↗
13. IoT type-of-traffic forecasting method based on gradient boosting neural networks10.1016/j.future.2019.12.013 ↗
14. Li, Z., Chen, X., Wu, L., Ahmed, A.-S., Wang, T., Zhang, Y., Li, H., Li, Z., Xu, Y., and Tong, Y. (2021). Error Analysis of Air-Core Coil Current Transformer Based on Stacking Model Fusion. Energies, 14.10.3390/en14071912 ↗
15. Load Forecasting Based on Multi-model by Stacking Ensemble Learning
16. A survey on ensemble learning10.1007/s11704-019-8208-z ↗
17. A bagging algorithm for the imputation of missing values in time series10.1016/j.eswa.2019.03.044 ↗
18. Online Bagging and Boosting for Imbalanced Data Streams10.1109/TKDE.2016.2609424 ↗
19. A New Appraisal Model of Second-Hand Housing Prices in China’s First-Tier Cities Based on Machine Learning Algorithms10.1007/s10614-020-09973-5 ↗
20. Strength of Stacking Technique of Ensemble Learning in Rockburst Prediction with Imbalanced Data: Comparison of Eight Single and Ensemble Models10.1007/s11053-020-09787-0 ↗
21. Wind power forecasting based on stacking ensemble model, decomposition and intelligent optimization algorithm10.1016/j.neucom.2021.07.084 ↗
22. Random projections as regularizers: Learning a linear discriminant from fewer observations than dimensions10.1007/s10994-014-5466-8 ↗
23. Detection of early stages of Alzheimer’s disease based on MEG activity with a ran-domized convolutional neural network10.1016/j.artmed.2020.101924 ↗
24. Hawkins, D.M. (1980). Identification of Outliers, Chapman and Hall.10.1007/978-94-015-3994-4 ↗
25. A gentle introduction to quantile regression for ecologists10.1890/1540-9295(2003)001[0412:AGITQR]2.0.CO;2 ↗
26. Multiple Imputation of Missing or Faulty Values Under Linear Constraints10.1080/07350015.2014.885435 ↗
27. Multiple imputation in the presence of high-dimensional data10.1177/0962280213511027 ↗
28. Residuals and regression diagnostics: Focusing on logistic regression10.21037/atm.2016.03.36 ↗
29. Identification and classification of multiple outliers, high leverage points and influential observations in linear regression10.1080/02664763.2015.1070806 ↗
30. Detection of Influential Observation in Linear Regression10.1080/00401706.1977.10489493 ↗
31. Belsley, D.A., Kuh, E., and Welsch, R.E. (2005). Regression Diagnostics: Identifying Influential Data and Sources of Collinearity, John Wiley & Sons.
32. Estimation of a type 2 Tobit model with generalized Box-Cox transformation10.1080/00036846.2020.1854667 ↗
33. Research on a Gas Concentration Prediction Algorithm Based on Stacking10.3390/s21051597 ↗
34. Detecting Novel Associations in Large Data Sets10.1126/science.1205438 ↗
35. Model selection for within-batch effect correction in UPLC-MS metabolomics using quality control—Support vector regression10.1016/j.aca.2018.04.055 ↗
36. Distance-based outliers: Algorithms and applications10.1007/s007780050006 ↗
37. Royston, P., and White, I. (2011). Multiple Imputation by Chained Equations (MICE): Implementation in Stata. J. Stat. Softw., 45.10.18637/jss.v045.i04 ↗
38. Buuren, S.V., and Oudshoorn, K. (1999). Flexible Mutlivariate Imputation by MICE, TNO.
39. Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., and Gulin, A. (2017). CatBoost: Unbiased boosting with categorical features. arXiv.
40. Dorogush, A.V., Ershov, V., and Gulin, A. (2018). CatBoost: Gradient boosting with categorical features support. arXiv.
41. Evaluation of CatBoost method for prediction of reference evapotranspiration in humid regions10.1016/j.jhydrol.2019.04.085 ↗
42. The use of linear regression to predict digestible protein and available amino acid contents of feed ingredients and diets for fish10.1016/j.aquaculture.2008.03.046 ↗
43. Practical selection of SVM parameters and noise estimation for SVM regression10.1016/S0893-6080(03)00169-2 ↗
44. Estimation of real-driving emissions for buses fueled with liquefied natural gas based on gradient boosted regression trees10.1016/j.scitotenv.2019.01.054 ↗
45. Chen, T., Tong, H., and Benesty, M. (2021, December 18). Xgboost: Extreme Gradient Boosting. Available online: https://github.com/dmlc/xgboost.
46. Machine learning with physicochemical relationships: Solubility prediction in organic solvents and water10.1038/s41467-020-19594-z ↗
47. Random forest10.1023/A:1010933404324 ↗
48. A fuzzy K-nearest neighbor algorithm10.1109/TSMC.1985.6313426 ↗
49. A comparative analysis of gradient boosting algorithms
50. Chen, T., and Guestrin, C. (2016, January 13–17). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference, San Francisco, CA, USA.10.1145/2939672.2939785 ↗
51. Classification of VHR Multispectral Images Using Extratrees and Maximally Stable Extremal Region-Guided Morphological Profile10.1109/JSTARS.2018.2824354 ↗
52. Study on short-term photovoltaic power prediction model based on the Stacking ensemble learning10.1016/j.egyr.2020.11.006 ↗
53. A Machine Learning (kNN) Approach to Predicting Global Seafloor Total Organic Carbon10.1029/2018GB005992 ↗
54. CatBoost for big data: An interdisciplinary review10.1186/s40537-020-00369-8 ↗
55. Evaluation of stacking and blending ensemble learning methods for estimating daily reference evapotranspiration10.1016/j.compag.2021.106039 ↗
56. Prediction of fishing vessel operation mode based on Stacking model fusion10.1088/1742-6596/1792/1/012030 ↗
57. Stacked generalization10.1016/S0893-6080(05)80023-1 ↗
58. Stacked regressions10.1007/BF00117832 ↗
59. Machine and K-Fold cross validation to predict compressive strength of concrete in marine environment10.1016/j.conbuildmat.2019.02.071 ↗
60. Akaike, H. (1998). Information Theory and an Extension of the Maximum Likelihood Principle, Springer.10.1007/978-1-4612-1694-0_15 ↗
61. Model Selection and Model Averaging in Phylogenetics: Advantages of Akaike Information Criterion and Bayesian Approaches over Likelihood Ratio Tests10.1080/10635150490522304 ↗
62. Fisher score and Matthews correlation coefficient-based feature subset selection for heart disease diagnosis using support vector machines10.1007/s10115-018-1185-y ↗
63. Effective use of Spearman’s and Kendall’s correlation coefficients for association between two measured traits10.1016/j.anbehav.2015.01.010 ↗
64. Bayesian classifiers based on kernel density estimation: Flexible classifiers10.1016/j.ijar.2008.08.008 ↗