A Taxonomy of Machine Learning Clustering Algorithms, Challenges, and Future Realms
Source abstract
Abstract
In the field of data mining, clustering has shown to be an important technique. Numerous clustering methods have been devised and put into practice, and most of them locate high-quality or optimum clustering outcomes in the field of computer science, data science, statistics, pattern recognition, artificial intelligence, and machine learning. This research provides a modern, thorough review of both classic and cutting-edge clustering methods. The taxonomy of clustering is presented in this review from an applied angle and the compression of some hierarchical and partitional clustering algorithms with various parameters. We also discuss the open challenges in clustering such as computational complexity, refinement of clusters, speed of convergence, data dimensionality, effectiveness and scalability, data object representation, evaluation measures, data streams, and knowledge extraction; scientists and professionals alike will be able to use it as a benchmark as they strive to advance the state-of-the-art in clustering techniques.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Clustering Algorithms Research · Data Stream Mining Techniques · Time Series Analysis and Forecasting
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer and Information Science Department (CISD), Universiti Teknologi PETRONAS, Bandar Seri Iskandar Perak 32610, Malaysia
Funding
Universiti Teknologi PETRONAS (UTP)
References (93)
Show source-provided references
1. Data clustering: 50 years beyond K-means10.1016/j.patrec.2009.09.011 ↗
2. Abualigah, L.M.Q. (2019). Feature Selection and Enhanced Krill Herd Algorithm for Text Document Clustering, Springer.10.1007/978-3-030-10674-4 ↗
3. Automatic data clustering using nature-inspired symbiotic organism search algorithm10.1016/j.knosys.2018.09.013 ↗
4. A new feature selection method to improve the document clustering using particle swarm optimization algorithm10.1016/j.jocs.2017.07.018 ↗
5. Tan, P.-N., Steinbach, M., and Kumar, V. (2018). Introduction to Data Mining Pearson Education India, Indian Nursing Council.
6. A robust dynamic niching genetic algorithm with niche migration for automatic clustering problem10.1016/j.patcog.2009.10.020 ↗
7. A novel combinatorial merge-split approach for automatic clustering using imperialist competitive algorithm10.1016/j.eswa.2018.09.050 ↗
8. Automatic clustering algorithms: A systematic review and bibliometric analysis of relevant literature10.1007/s00521-020-05395-4 ↗
9. Plant, C., and Böhm, C. (2010). Evolving Application Domains of Data Warehousing and Mining: Trends and Solutions, IGI Global.
10. A survey of clustering algorithms for an industrial context10.1016/j.procs.2019.01.022 ↗
11. Dryden, N., Böhringer, R., Ben-Nun, T., and Hoefler, T. (2021, January 14–19). Clairvoyant prefetching for distributed machine learning I/O. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, St. Louis, MO, USA.10.1145/3458817.3476181 ↗
12. Khan, A., Paul, A.K., Zimmer, C., Oral, S., Dash, S., Atchley, S., and Wang, F. (2022, January 5–8). Hvac: Removing I/O Bottleneck for Large-Scale Deep Learning Applications. Proceedings of the 2022 IEEE International Conference on Cluster Computing (CLUSTER), Heidelberg, Germany.10.1109/CLUSTER51413.2022.00044 ↗
13. A survey of clustering algorithms for big data: Taxonomy and empirical analysis10.1109/TETC.2014.2330519 ↗
14. Berkhin, P., Beche, J.D., and Randall, D.J. (2001, January 26–29). Interactive path analysis of web site traffic. Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/502512.502574 ↗
15. A review of clustering techniques and developments10.1016/j.neucom.2017.06.053 ↗
16. A survey on parallel clustering algorithms for big data10.1007/s10462-020-09918-2 ↗
17. Sharif, Z., Jung, L.T., Ayaz, M., and Yahya, M. (2023). Priority-based Task Scheduling and Resource Allocation in Edge Computing for Health Monitoring System. J. King Saud Univ. Comput. Inf. Sci.10.1016/j.jksuci.2023.01.001 ↗
18. Nagpal, A., Jatain, A., and Gaur, D. (2013, January 11–12). Review based on data clustering algorithms. Proceedings of the 2013 IEEE Conference on Information & Communication Technologies, Thuckalay, India.10.1109/CICT.2013.6558109 ↗
19. Clustering algorithms: Their application to gene expression data10.4137/BBI.S38316 ↗
20. Jain, A.K., and Dubes, R.C. (1988). Algorithms for Clustering Data, Prentice-Hall, Inc.
21. Sharif, Z., Jung, L.T., Ayaz, M., Yahya, M., and Pitafi, S. (2022, January 1–2). A Taxonomy for Resource Management in Edge Computing, Applications and Future Realms. Proceedings of the 2022 International Conference on Digital Transformation and Intelligence (ICDI), Kuching, Sarawak, Malaysia.10.1109/ICDI57181.2022.10007397 ↗
22. Nature-inspired metaheuristic techniques for automatic clustering: A survey and performance study10.1007/s42452-020-2073-0 ↗
23. Bindra, K., and Mishra, A. (2017, January 20–22). A detailed study of clustering algorithms. Proceedings of the 2017 6th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions)(ICRITO), Noida, India.10.1109/ICRITO.2017.8342454 ↗
24. Djouzi, K., and Beghdad-Bey, K. (2019, January 26–27). A review of clustering algorithms for big data. Proceedings of the 2019 International Conference on Networking and Advanced Systems (ICNAS), Annaba, Algeria.10.1109/ICNAS.2019.8807822 ↗
25. A comprehensive survey of clustering algorithms10.1007/s40745-015-0040-1 ↗
26. Review of Clustering Techniques in Control System: Review of Clustering Techniques in Control System10.1016/j.procs.2020.06.032 ↗
27. Algorithms for hierarchical clustering: An overview10.1002/widm.53 ↗
28. Parallel algorithms for hierarchical clustering10.1016/0167-8191(95)00017-I ↗
29. Data clustering: A review10.1145/331499.331504 ↗
30. A survey of algorithms for contiguity-constrained clustering and related problems10.1093/comjnl/28.1.82 ↗
31. Sharif, Z., Jung, L.T., and Ayaz, M. (2022, January 25–27). Priority-based Resource Allocation Scheme for Mobile Edge Computing. Proceedings of the 2022 2nd International Conference on Computing and Information Technology (ICCIT), Tabuk, Saudia Arabia.10.1109/ICCIT52419.2022.9711641 ↗
32. Müllner, D. (2011). Modern hierarchical, agglomerative clustering algorithms. arXiv.
33. Agglomerative clustering using the concept of mutual nearest neighbourhood10.1016/0031-3203(78)90018-3 ↗
34. Rathore, P. (2018). Big Data Cluster Analysis and Its Applications. [Ph.D. Thesis, University of Melbourne].
35. Survey of clustering algorithms10.1109/TNN.2005.845141 ↗
36. SLINK: An optimally efficient algorithm for the single-link cluster method10.1093/comjnl/16.1.30 ↗
37. An efficient algorithm for a complete link method10.1093/comjnl/20.4.364 ↗
38. Implementing agglomerative hierarchic clustering algorithms for use in document retrieval10.1016/0306-4573(86)90097-X ↗
39. A survey of recent advances in hierarchical clustering algorithms10.1093/comjnl/26.4.354 ↗
40. Efficient algorithms for agglomerative hierarchical clustering methods10.1007/BF01890115 ↗
41. Sharif, Z., Jung, L.T., Razzak, I., and Alazab, M. (2021). Adaptive and priority-based resource allocation for efficient resources utilization in mobile edge computing. IEEE Internet Things J.
42. Savaresi, S.M., Boley, D.L., Bittanti, S., and Gazzaniga, G. (2002, January 11–13). Cluster Selection in Divisive Clustering Algorithms. Proceedings of the 2002 SIAM International Conference on Data Mining, Arlington, VA, USA.10.1137/1.9781611972726.18 ↗
43. Principal direction divisive partitioning10.1023/A:1009740529316 ↗
44. DIVCLUS-T: A monothetic divisive hierarchical clustering method10.1016/j.csda.2007.03.013 ↗
45. Karypis, G., and Kumar, V. (1999, January 21–25). Multilevel k-way hypergraph partitioning. Proceedings of the 36th Annual Acm/Ieee Design Automation Conference, New Orleans, LA, USA.10.1109/DAC.1999.781339 ↗
46. DIVFRP: An automatic divisive hierarchical clustering method based on the furthest reference points10.1016/j.patrec.2008.07.002 ↗
47. A fast divisive clustering algorithm using an improved discrete particle swarm optimizer10.1016/j.patrec.2010.04.001 ↗
48. Dissimilarity analysis: A new technique of hierarchical sub-division10.1038/2021034a0 ↗
49. Kaufman, L., and Rousseeuw, P.J. (2009). Finding Groups in Data: An Introduction to Cluster Analysis, John Wiley & Sons.
50. The weighted sum of split and diameter clustering10.1007/BF01246100 ↗
51. Efficient algorithms for divisive hierarchical clustering with the diameter criterion10.1007/BF02616245 ↗
52. Thirty years of numerical taxonomy10.2307/2413593 ↗
53. Multivariate methods in plant ecology: I. Association-analysis in plant communities10.2307/2257249 ↗
54. Brito, P.M., and Chavent, M. (2012, January 6–8). Divisive Monothetic Clustering for Interval and Histogram-Valued Data. Proceedings of the ICPRAM 2012-1st International Conference on Pattern Recognition Applications and Methods, Algarve, Portugal.
55. Zhu, J. (2019). Divisive Hierarchical Clustering for Interval-Valued Data. [Ph.D. Thesis, University of Georgia].
56. A polythetic clustering process and cluster validity indexes for histogram-valued objects10.1016/j.csda.2011.01.011 ↗
57. A k-mean clustering algorithm for mixed numeric and categorical data10.1016/j.datak.2007.03.016 ↗
58. Dinh, D.-T., Fujinami, T., and Huynh, V.-N. (2019, January 29). Estimating the Optimal Number of Clusters in Categorical Data Clustering By Silhouette Coefficient. Proceedings of the International Symposium on Knowledge and Systems Sciences, Singapore.10.1007/978-981-15-1209-4_1 ↗
59. Fuzzy sets
60. Subspace clustering10.1109/MSP.2010.939739 ↗
61. Subspace clustering for high dimensional data: A review10.1145/1007730.1007731 ↗
62. Deshmukh, H.S., and Ramteke, P. (2015). Comparing the Techniques of Cluster Analysis for Big Data. Int. J. Adv. Res. Comput. Eng. Technol., 4.
63. A generalized histogram clustering scheme for multidimensional image data10.1016/0031-3203(83)90022-5 ↗
64. Leland McInnes, J.H., and Astels, S. (2023, January 11). Comparing Clustering Algorithms Using Python. Available online: https://hdbscan.readthedocs.io/en/latest/comparing_clustering_algorithms.html#hdbscan.
65. Clustering methods for Big data analysis
66. How many clusters? Which clustering method? Answers via model-based cluster analysis10.1093/comjnl/41.8.578 ↗
67. Agarwal, P., Alam, M.A., and Biswas, R. (2011). Issues, challenges and tools of clustering algorithms. arXiv.
68. Automatic clustering using nature-inspired metaheuristics: A survey10.1016/j.asoc.2015.12.001 ↗
69. Automatic subspace clustering of high dimensional data10.1007/s10618-005-1396-1 ↗
70. Ray, S. (2019, January 14–16). A Quick Review of Machine Learning Algorithms. Proceedings of the 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), Faridabad, India.10.1109/COMITCon.2019.8862451 ↗
71. Algorithm AS 136: A k-means clustering algorithm
72. MacQueen, J. (1967, January 18–21). Classification and Analysis of Multivariate Observations. Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA.
73. Clustering of time series data—A survey10.1016/j.patcog.2005.01.025 ↗
74. An evaluation of data stream clustering algorithms10.1002/sam.11380 ↗
75. Gama, J., and Gaber, M.M. (2007). Learning from Data Streams: Processing Techniques in Sensor Networks, Springer.10.1007/3-540-73679-4 ↗
76. Mean shift: A robust approach toward feature space analysis10.1109/34.1000236 ↗
77. The estimation of the gradient of a density function, with applications in pattern recognition10.1109/TIT.1975.1055330 ↗
78. Sasaki, H., Kanamori, T., Hyvärinen, A., Niu, G., and Sugiyama, M. (2018). Mode-seeking clustering and density ridge estimation via direct estimation of density-derivative-ratios. J. Mach. Learn. Res., 17–380.
79. Duin, R.P., Fred, A.L., Loog, M., and Pękalska, E. (2012, January 7–9). Mode seeking clustering by KNN and mean shift evaluated. Proceedings of the Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR), Hiroshima, Japan.10.1007/978-3-642-34166-3_6 ↗
80. Carreira-Perpinán, M.A. (2015). A review of mean-shift algorithms for clustering. arXiv.
81. Robust clustering using a kNN mode seeking ensemble10.1016/j.patcog.2017.11.023 ↗
82. Estimation and hypothesis testing in finite mixture models10.1111/j.2517-6161.1985.tb01331.x ↗
83. Marriott, F.H.C. (1974). Interpretation of Multiple Observations, Academic Press.
84. Unsupervised and semi-supervised clustering: A brief survey
85. A review of the expectation maximization algorithm in data-driven process identification10.1016/j.jprocont.2018.12.010 ↗
86. Verbeek, J. (2004). Mixture Models for Clustering and Dimension Reduction. [Ph.D. Thesis, Universiteit van Amsterdam].
87. Das, S., Abraham, A., and Konar, A. (2008). Computational Intelligence in Bioinformatics, Springer.
88. Shirkhorshidi, A.S., Aghabozorgi, S., Wah, T.Y., and Herawan, T. (July, January 30). Big data clustering: A review. Proceedings of the International Conference on Computational Science and Its Applications, Guimarães, Portugal.
89. Zerhari, B., Lahcen, A.A., and Mouline, S. (2015, January 25–26). Big data clustering: Algorithms and challenges. Proceedings of the International Conference on Big Data, Cloud and Applications (BDCA’15), Tetuan, Morocco.
90. Wang, Z., Chang, S., Zhou, J., Wang, M., and Huang, T.S. (2016, January 5–7). Learning a Task-Specific Deep Architecture for Clustering. Proceedings of the 2016 SIAM International Conference on Data Mining, Miami, FL, USA.10.1137/1.9781611974348.42 ↗
91. Clustering mixed numerical and categorical data with missing values10.1016/j.ins.2021.04.076 ↗
92. Bouveyron, C., Hammer, B., and Villmann, T. (2012, January 25–27). Recent developments in clustering algorithms. Proceedings of the ESANN, Bruges, Belgium.
93. Kokate, U., Deshpande, A., Mahalle, P., and Patil, P. (2018). Data stream clustering techniques, applications, and models: Comparative analysis and discussion. Big Data Cogn. Comput., 2.10.3390/bdcc2040032 ↗