Self-Tuning Parameters for Decision Tree Algorithm Based on Big Data Analytics
Source abstract
Abstract
Big data is usually unstructured, and many applications require the analysis in real-time. Decision tree (DT) algorithm is widely used to analyze big data. Selecting the optimal depth of DT is time-consuming process as it requires many iterations. In this paper, we have designed a modified version of a (DT). The tree aims to achieve optimal depth by self-tuning running parameters and improving the accuracy. The efficiency of the modified (DT) was verified using two datasets (airport and fire datasets). The airport dataset has 500000 instances and the fire dataset has 600000 instances. A comparison has been made between the modified (DT) and standard (DT) with results showing that the modified performs better. This comparison was conducted on multi-node on Apache Spark tool using Amazon web services. Resulting in accuracy with an increase of 6.85% for the first dataset and 8.85% for the airport dataset. In conclusion, the modified DT showed better accuracy in handling different-sized datasets compared to standard DT algorithm.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Big Data and Business Intelligence · Data Mining and Machine Learning Applications · Artificial Intelligence in Healthcare
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (31)
Show source-provided references
1. Using decision tree to select forecasting algorithms in distinct electricity consumption context of an office building10.1016/j.egyr.2022.01.046 ↗
2. Predicting ambulance offload delay using a hybrid decision tree model10.1016/j.seps.2021.101146 ↗
3. A comparative performance study of Naïve and ensemble algorithms for E-commerce
4. Mining patient opinion to evaluate the service quality in healthcare: A deep-learning approach10.1007/s12652-019-01434-8 ↗
5. What patients like or dislike in physicians: Analyzing drivers of patient satisfaction and dissatisfaction using a digital topic modeling approach10.1016/j.ipm.2021.102516 ↗
6. Does export product diversification spur energy demand in the APEC region? Application of a new neural networks experiment and a decision tree model10.1016/j.enbuild.2021.111820 ↗
7. Fault zone identification and phase selection for microgrids using decision trees ensemble10.1016/j.ijepes.2021.107178 ↗
8. Short term effect evaluation model of rural energy construction revitalization based on ID3 decision tree algorithm10.1016/j.egyr.2022.01.239 ↗
9. Performance evaluation among ID3, C4.5, and CART decision tree algorithm10.1007/978-981-16-5640-8_11 ↗
10. Rule formation application based on C4. 5 algorithm for household electricity usage prediction10.48048/tis.2022.2167 ↗
11. Information gain-based exploration using rao-blackwellized particle filters
12. Decision trees for binary classification variables grow equally with the Gini impurity measure and Pearson’s chi-square test10.1504/IJBIDM.2007.013938 ↗
13. Classification based on decision tree algorithm for machine learning10.38094/jastt20165 ↗
14. Risk factors, time to onset and recurrence of delirium in a mixed medical-surgical ICU population: A secondary analysis using Cox and CHAID decision tree modeling
15. The quest for customer intelligence to support marketing decisions: A knowledge-based framework10.1142/S2196888822500208 ↗
16. Is decision tree obsolete in its original form? A burning debate10.18280/ria.360112 ↗
17. A novel surrogate model for channel geometry optimization of PEM fuel cell based on bagging-SVM ensemble regression10.1016/j.ijhydene.2022.02.239 ↗
18. Enhancing the performance of decision tree: A research study of dealing with unbalanced data
19. Improved email spam classification method using integrated particle swarm optimization and decision tree
20. An enhanced approach for classifying emotions using customized decision tree algorithm
21. Data-driven cervical cancer prediction model with outlier detection and over-sampling methods10.3390/s20102809 ↗
22. Dynamical properties of ion-acoustic waves in space plasma and its application to image encryption10.1109/ACCESS.2021.3054250 ↗
23. A personalized healthcare monitoring system for diabetic patients by utilizing BLE-based sensors and real-time data processing10.3390/s18072183 ↗
24. Hybrid prediction model for type 2 diabetes and hypertension using DBSCAN-based outlier detection, synthetic minority over sampling technique (SMOTE), and random forest10.3390/app8081325 ↗
25. Effective selection of machine learning algorithms for big data analytics using apache spark
26. Efficient large scale nlp feature engineering with apache spark
27. A survey of scheduling tasks in big data: Apache spark10.1007/978-981-16-8721-1_39 ↗
28. WS-PDC: Persistent distributed channel-based web services applied on IFRS data processing and loading
29. Survey of distributed file system design choices10.1145/3465405 ↗
30. The google file system
31. Comparative analysis of google file system and hadoop distributed file system