SMConf: One-Size-Fit-Bunch, Automated Memory Capacity Configuration for In-memory Data Analytic Platform
Source abstract
Abstract
Spark is the most popular in-memory processing framework for big data analytics. Memory is the crucial resource for workloads to achieve performance acceleration on Spark. The extant memory capacity configuration approach in ... | Find, read and cite all the research you need on Tech Science Press
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Cloud Computing and Resource Management · Data Stream Mining Techniques · Advanced Data Storage Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (45)
Show source-provided references
1. A hierarchical distributed processing framework for big image data10.1109/TBDATA.2016.2613992 ↗
2. ClimateSpark: An in-memory distributed computing framework for big climate data analytics10.1016/j.cageo.2018.03.011 ↗
3. Spark: Cluster computing with working sets
4. An experimental analysis of limitations of MapReduce for iterative algorithms on Spark10.1007/s10586-017-1167-y ↗
5. MEMTUNE: Dynamic memory management for in-memory data analytic platforms
6. Shanmugam · Best Practices for Successfully Managing Memory for Apache Spark applications on Amazon EMR · 2019
7. Dynamic memory-aware scheduling in Spark computing environment10.1016/j.jpdc.2020.03.010 ↗
8. Performance Tuning for the Spark Engine · 2020
9. Heterogeneity and dynamicity of clouds at scale: Google trace analysis
10. Barroso · The Datacenter as a Computer: Designing Warehouse-Scale Machines · 2018 · 1
11. BestConfig: tapping the performance potential of systems via automatic configuration tuning
12. SparkBench: A comprehensive benchmarking suite for in memory data analytic platform Spark
13. The PARSEC benchmark suite: Characterization and architectural implications
14. Understanding the behavior of in-memory computing workloads
15. Patterson · Computer A · 2019 · 1
16. A Pearson’s correlation coefficient based decision tree and its parallel implementation10.1016/j.ins.2017.12.059 ↗
17. Meng · Annals of Statistics · 1994 · 22 · 1142
18. A simple and fast algorithm for k-medoids clustering10.1016/j.eswa.2008.01.039 ↗
19. Compositional dissimilarity as a robust measure of ecological distance10.1007/BF00038687 ↗
20. Distance metric learning for large margin nearest neighbor classification
21. Stepwise selection in small data sets: A simulation study of bias in logistic regression analysis10.1016/S0895-4356(99)00103-1 ↗
22. On the noise model of support vector machine regression
23. LIBSVM: A library for support vector machines10.1145/1961189.1961199 ↗
24. A parallel random forest algorithm for big data in a Spark cloud computing environment10.1109/TPDS.2016.2603511 ↗
25. Performance prediction for Apache Spark platform
26. Distributed data analytics platform for wide-area synchrophasor measurement systems10.1109/TSG.2016.2528895 ↗
27. Low latency analytics for streaming traffic data with Apache Spark
28. A Spark scheduling strategy for heterogeneous cluster
29. DSpark: Deadline-based resource allocation for big data applications in Apache Spark
30. CSAS: Cost-based storage auto-selection, a fine grained storage selection mechanism for Spark
31. Towards automatic memory tuning for in-memory big data analytics in clusters
32. Automatic caching decision for scientific dataflow execution in Apache Spark
33. Reference-distance eviction and prefetching for cache management in Spark
34. Intelligent RDD management for high performance in-memory computing in Spark
35. JeCache: Just-enough data caching with just-in-time prefetching for big data applications
36. Optimizing shuffle in wide-area data analytics
37. A dynamic memory allocation optimization mechanism based on Spark
38. Taming non-local stragglers using efficient prefetching in MapReduce
39. An improved memory cache management study based on Spark
40. A novel method for tuning configuration parameters of Spark based on machine learning
41. Understanding the influence of configuration settings: An execution model-driven framework for Apache Spark platform
42. Auto-tuning Spark configurations based on neural network
43. Towards automatic tuning of Apache Spark configuration
44. Characterizing data analysis workloads in data centers
45. An optimization of virtual machine selection and placement by using memory content similarity for server consolidation in cloud10.1016/j.future.2018.02.026 ↗