Unbiased Isotonic Regression Tree for Discovering Hidden Heterogeneity in Monotonicity Constraints
Source abstract
Abstract
Integrating domain knowledge is increasingly recognized as vital for improving the relevance and reliability of machine learning models. This integration is often implemented through specific types of constraints that reflect real-world conditions or theoretical insights. Within the family of regression trees, the isotonic regression tree is used to incorporate a monotonicity constraint between a predictor and a response variable. However, the isotonic regression tree could be susceptible to split selection bias, as it selects both its split variable and cutpoint simultaneously. This study first explores the possibility of selection bias on split variables in the isotonic regression tree and proposes an unbiased isotonic regression tree that mitigates the issue of the selection bias problem. The results of the simulation and case study demonstrate the effectiveness of the proposed approach and the ability to discover hidden heterogeneous monotonic constraints.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning and Data Classification · Neural Networks and Applications · Face and Expression Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Data Science, Inha University, Incheon 22212, Republic of Korea
Funding
INHA UNIVERSITY
References (61)
Show source-provided references
1. Analysis of various decision tree algorithms for classification in data mining
2. Classification and regression tree with resampling for classifying imbalanced data
3. Reconfigured piecewise linear regression tree for multistage manufacturing process control10.1080/0740817X.2011.564603 ↗
4. A decision tree method for building energy demand modeling10.1016/j.enbuild.2010.04.006 ↗
5. Applying the regression tree method to predict students’ science achievement10.9788/s43076-019-00002-5 ↗
6. Tree-structured subgroup analysis of receiver operating characteristic curves for diagnostic tests10.1016/j.acra.2012.09.007 ↗
7. Baker, N., Alexander, F., Bremer, T., Hagberg, A., Kevrekidis, Y., Najm, H., Parashar, M., Patra, A., Sethian, J., and Wild, S. (2019). Workshop Report on Basic Research Needs for Scientific Machine Learning: Core Technologies for Artificial Intelligence, USDOE Office of Science (SC).10.2172/1478744 ↗
8. Shape constrained additive models10.1007/s11222-013-9448-7 ↗
9. Constrained hierarchical modeling of degradation data in tissue-engineered scaffold fabrication10.1080/0740817X.2015.1019164 ↗
10. Wakasugi, K. (2021, January 19–26). Sensitivity Direction Learning with Neural Networks Using Domain Knowledge as Soft Shape Constraints. Proceedings of the IJCAI International Joint Conference on Artificial Intelligence, Montreal, QC, Canada.10.24963/ijcai.2021/422 ↗
11. Bayesian monotone regression using Gaussian process projection10.1093/biomet/ast063 ↗
12. Estimating shape constrained functions using Gaussian processes10.1137/140955033 ↗
13. Bayesian analysis of shape-restricted functions using Gaussian process priors
14. Constrained Gaussian process with application in tissue-engineering scaffold biodegradation10.1080/24725854.2017.1414973 ↗
15. Runje, D., and Shankaranarayana, S.M. (2023, January 23–29). Constrained monotonic neural networks. Proceedings of the International Conference on Machine Learning, Honolulu, HI, USA.
16. Mitchell, T.M. (1980). The Need for Biases in Learning Generalizations, Rutgers University.
17. Incorporating second-order functional knowledge for better option pricing
18. Veličković, P. (2019). The Resurgence of Structure in Deep Neural Networks, University of Cambridge.
19. Monotonic calibrated interpolated look-up tables
20. Certified monotonic neural networks
21. Market segmentation trees10.1287/msom.2023.1195 ↗
22. Brunk, H.D. (1969). Estimation of Isotonic Regression, University of Missouri-Columbia.
23. On consistency in monotonic regression10.1214/aos/1176342407 ↗
24. Breiman, L. (2017). Classification and Regression Trees, Routledge.10.1201/9781315139470 ↗
25. Fifty years of classification and regression trees10.1111/insr.12016 ↗
26. Problems in the analysis of survey data, and a proposal10.1080/01621459.1963.10500855 ↗
27. Piecewise-polynomial regression trees
28. Regression tress with unbiased variable selection and interaction detection
29. Constraint enforcement on decision trees: A survey10.1145/3506734 ↗
30. What makes classification trees comprehensible?10.1016/j.eswa.2016.06.009 ↗
31. Optimal classification trees10.1007/s10994-017-5633-9 ↗
32. Column generation based heuristic for learning classification trees10.1016/j.cor.2019.104866 ↗
33. Deep lattice networks and partial monotonic functions
34. Rank entropy-based decision trees for monotonic classification10.1109/TKDE.2011.149 ↗
35. Information entropy for ordinal classification10.1007/s11432-010-3117-7 ↗
36. Rank discrimination measures for enforcing monotonicity in decision tree induction10.1016/j.ins.2014.08.045 ↗
37. Multivariate decision trees with monotonicity constraints10.1016/j.knosys.2016.08.023 ↗
38. Kachuee, M., Karkkainen, K., Goldstein, O., Zamanzadeh, D., and Sarrafzadeh, M. (2019). Nutrition and health data for cost-sensitive learning. arXiv.
39. Cost-sensitive classification: Empirical evaluation of a hybrid genetic decision tree induction algorithm10.1613/jair.120 ↗
40. Decision tree classification with differential privacy: A survey
41. Friedman, A., and Schuster, A. (2010, January 24–28). Data mining with differential privacy. Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA.10.1145/1835804.1835868 ↗
42. Kamiran, F., Calders, T., and Pechenizkiy, M. (2010, January 13). Discrimination aware decision tree learning. Proceedings of the 2010 IEEE International Conference on Data Mining, Sydney, Australia.10.1109/ICDM.2010.50 ↗
43. Raff, E., Sylvester, J., and Mills, S. (2018, January 1–3). Fair forests: Regularized tree induction to minimize model bias. Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, New Orleans, LA, USA.10.1145/3278721.3278742 ↗
44. Zafar, M.B., Valera, I., Rogriguez, M.G., and Gummadi, K.P. (2017, January 20–22). Fairness constraints: Mechanisms for fair classification. Proceedings of the Artificial Intelligence and Statistics, Ft. Lauderdale, FL, USA.
45. Struyf, J., and Džeroski, S. (2007, January 17–21). Clustering trees with instance level constraints. Proceedings of the Machine Learning: ECML 2007: 18th European Conference on Machine Learning, Warsaw, Poland.
46. Papernot, N., McDaniel, P., and Goodfellow, I. (2016). Transferability in machine learning: From phenomena to black-box attacks using adversarial samples. arXiv.
47. Jordan, A.I., Mühlemann, A., and Ziegel, J.F. (2019). Optimal solutions to the isotonic regression problem. arXiv.
48. Algorithms for a class of isotonic regression problems10.1007/PL00009258 ↗
49. Barlow, R.E. (1972). Statistical Inference Under Order Restrictions: The Theory and Application of Isotonic Regression, J. Wiley.
50. Jin, R., and Agrawal, G. (2003, January 1–3). Communication and memory efficient parallel decision tree construction. Proceedings of the 2003 SIAM International Conference on Data Mining, San Francisco, CA, USA.10.1137/1.9781611972733.11 ↗
51. Mcrank: Learning to rank using multiple classification and gradient boosting
52. Ranka, S., and Singh, V. (1998, January 27–31). CLOUDS: A decision tree classifier for large datasets. Proceedings of the 4th Knowledge Discovery and Data Mining Conference, New York, NY, USA.
53. Model-based recursive partitioning10.1198/106186008X319331 ↗
54. A recursive partitioning approach for subgroup identification in brain–behaviour correlation analysis10.1007/s10044-018-00775-y ↗
55. Ding, Y. (2019). Data Science for Wind Energy, Chapman and Hall/CRC.10.1201/9780429490972 ↗
56. Wind tunnel testing of a closed-loop wake deflection controller for wind farm power maximization10.1088/1742-6596/753/3/032006 ↗
57. A production economics analysis for quantifying the efficiency of wind turbines10.1002/we.2105 ↗
58. Algorithm AS 302: Multiple Isotonic Regression
59. Isotonic regression for multiple independent variables10.1007/s00453-013-9814-z ↗
60. A variational local weighted deep sub-domain adaptation network for remaining useful life prediction facing cross-domain condition10.1016/j.ress.2022.108986 ↗
61. Lifetime extension approach based on the Levenberg–Marquardt neural network and power routing of DC–DC converters10.1109/TPEL.2023.3275791 ↗