Swarm Intelligence Algorithms for Feature Selection: A Review
Source abstract
Abstract
The increasingly rapid creation, sharing and exchange of information nowadays put researchers and data scientists ahead of a challenging task of data analysis and extracting relevant information out of data. To be able to learn from data, the dimensionality of the data should be reduced first. Feature selection (FS) can help to reduce the amount of data, but it is a very complex and computationally demanding task, especially in the case of high-dimensional datasets. Swarm intelligence (SI) has been proved as a technique which can solve NP-hard (Non-deterministic Polynomial time) computational problems. It is gaining popularity in solving different optimization problems and has been used successfully for FS in some applications. With the lack of comprehensive surveys in this field, it was our objective to fill the gap in coverage of SI algorithms for FS. We performed a comprehensive literature review of SI algorithms and provide a detailed overview of 64 different SI algorithms for FS, organized into eight major taxonomic categories. We propose a unified SI framework and use it to explain different approaches to FS. Different methods, techniques, and their settings are explained, which have been used for various FS aspects. The datasets used most frequently for the evaluation of SI algorithms for FS are presented, as well as the most common application areas. The guidelines on how to develop SI approaches for FS are provided to support researchers and analysts in their data mining tasks and endeavors while existing issues and open questions are being discussed. In this manner, using the proposed framework and the provided explanations, one should be able to design an SI approach to be used for a specific FS problem.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Metaheuristic Optimization Algorithms Research · Artificial Immune Systems Applications · Evolutionary Algorithms and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Institute of Informatics, Faculty of Electrical Engineering and Computer Science, University of Maribor, Koroška cesta 46, 2000 Maribor, Slovenia
References (100+)
Show source-provided references
1. Cao, J., Cui, H., Shi, H., and Jiao, L. (2016). Big Data: A Parallel Particle Swarm Optimization-Back-Propagation Neural Network Algorithm Based on MapReduce. PLoS ONE, 11.10.1371/journal.pone.0157551 ↗
2. Yin, H., Tang, K., Gao, Y., Klawonn, F., Lee, M., Weise, T., Li, B., and Yao, X. (2013). Swarm Intelligence in Big Data Analytics. Intelligent Data Engineering and Automated Learning—IDEAL 2013, Springer.10.1007/978-3-642-41278-3 ↗
3. Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer. [2nd ed.].10.1007/978-0-387-84858-7 ↗
4. Blum, C., and Li, X. (2008). Swarm Intelligence in Optimization. Swarm Intelligence, Springer.10.1007/978-3-540-74089-6 ↗
5. Hassanien, A.E., and Emary, E. (2016). Swarm Intelligence: Principles, Advances, and Applications, CRC Press.10.1201/b19133 ↗
6. Garey, M.R., and Johnson, D.S. (1979). Computers and Intractability, a Guide to the Theory of NP-Completness, W. H. Freeman & Co.
7. Kennedy, J., and Eberhart, R. (December, January 27). Particle swarm optimization. Proceedings of the IEEE International Conference on Neural Networks (ICNN ’95), Perth, WA, Australia.
8. Dorigo, M. (1992). Learning and Natural Algorithms. [Ph.D. Thesis, Politecnico di Milano].
9. New inspirations in swarm intelligence: A survey10.1504/IJBIC.2011.038700 ↗
10. Bio inspired computing—A review of algorithms and scope of applications10.1016/j.eswa.2016.04.018 ↗
11. Feature Selection in Life Science Classification: Metaheuristic Swarm Search10.1109/MITP.2014.50 ↗
12. Comparison on Swarm Algorithms for Feature Selections Reductions
13. A Survey on Evolutionary Computation Approaches to Feature Selection10.1109/TEVC.2015.2504420 ↗
14. Dario, P., Sandini, G., and Aebischer, P. (1993). Swarm Intelligence in Cellular Robotic Systems. Robots and Biological Systems: Towards a New Bionics?, Springer.10.1007/978-3-642-58069-7 ↗
15. Engelbrecht, A.P. (2007). Computational Intelligence: An introduction, Wiley.10.1002/9780470512517 ↗
16. Millonas, M.M. (arXiv, 1993). Swarms, Phase Transitions, and Collective Intelligence, arXiv.
17. Olariu, S., and Zomaya, A.Y. (2005). Handbook of Bioinspired Algorithms and Applications, Chapman and Hall/CRC.10.1201/9781420035063 ↗
18. A Brief Review of Nature-Inspired Algorithms for Optimization
19. Monkey search: A novel metaheuristic search for global optimization10.1063/1.2817338 ↗
20. A new optimization method: Dolphin echolocation10.1016/j.advengsoft.2013.03.004 ↗
21. Chen, C.C., Tsai, Y.C., Liu, I.I., Lai, C.C., Yeh, Y.T., Kuo, S.Y., and Chou, Y.H. (2015, January 9–12). A Novel Metaheuristic: Jaguar Algorithm with Learning Behavior. Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Kowloon, China.10.1109/SMC.2015.282 ↗
22. Beyerer, J., Niggemann, O., and Kühnert, C. (2017). Agent Swarm Optimization: Exploding the search space. Machine Learning for Cyber Physical Systems, Springer.10.1007/978-3-662-53806-7 ↗
23. Water Distribution System Computer-Aided Design by Agent Swarm Optimization10.1111/mice.12062 ↗
24. Toward integrating feature selection algorithms for classification and clustering10.1109/TKDE.2005.66 ↗
25. Tang, J., Alelyani, S., and Liu, H. (2014). Feature selection for classification: A review. Data Classification: Algorithms and Applications, CRC Press.
26. Liu, H., and Motoda, H. (2007). Computational Methods of Feature Selection, CRC Press.10.1201/9781584888796 ↗
27. Wrappers for feature subset selection10.1016/S0004-3702(97)00043-X ↗
28. Brezočnik, L. (2017, January 6–8). Feature Selection for Classification Using Particle Swarm Optimization. Proceedings of the 17th IEEE International Conference on Smart Technologies (IEEE EUROCON 2017), Ohrid, Macedonia.10.1109/EUROCON.2017.8011255 ↗
29. Particle swarm optimization for parameter determination and feature selection of support vector machines10.1016/j.eswa.2007.08.088 ↗
30. Modified binary PSO for feature selection using SVM applied to mortality prediction of septic patients10.1016/j.asoc.2013.03.021 ↗
31. Application of global optimization methods to model and feature selection10.1016/j.patcog.2012.04.015 ↗
32. Xue, B., Zhang, M., and Browne, W.N. (2013). Novel Initialisation and Updating Mechanisms in PSO for Feature Selection in Classification. Applications of Evolutionary Computation: 16th European Conference, EvoApplications 2013, Springer.10.1007/978-3-642-37192-9_43 ↗
33. Image steganalysis using a bee colony based feature selection algorithm10.1016/j.engappai.2013.09.016 ↗
34. Binary grey wolf optimization approaches for feature selection10.1016/j.neucom.2015.06.083 ↗
35. An advanced ACO algorithm for feature subset selection10.1016/j.neucom.2014.06.067 ↗
36. Feature Selection Using Ant Colony Optimization (ACO): A New Method and Comparative Study in the Application of Face Recognition System10.1007/978-3-540-73435-2_6 ↗
37. A Modified Ant Colony Optimization Algorithm for Tumor Marker Gene Selection10.1016/S1672-0229(08)60050-9 ↗
38. Data feature selection based on Artificial Bee Colony algorithm10.1186/1687-5281-2013-47 ↗
39. Abraham, A., Krömer, P., and Snasel, V. (2015). Feature Subset Selection Approach by Gray-Wolf Optimization. Afro-European Conference for Industrial Advancement: Proceedings of the First International Afro-European Conference for Industrial Advancement AECIA 2014, Springer.10.1007/978-3-319-13572-4 ↗
40. Emary, E., Zawbaa, H.M., and Grosan, C. (2017). Experienced Gray Wolf Optimization Through Reinforcement Learning and Neural Networks. IEEE Trans. Neural Netw. Learn. Syst., 1–14.
41. Marinakis, Y., Marinaki, M., and Matsatsinis, N. (2009, January 6–9). A hybrid discrete Artificial Bee Colony—GRASP algorithm for clustering. Proceedings of the 2009 International Conference on Computers & Industrial Engineering, Troyes, France.10.1109/ICCIE.2009.5223810 ↗
42. Optimal feature selection using distance-based discrete firefly algorithm with mutual information criterion10.1007/s00521-016-2204-0 ↗
43. Gray Wolf Optimizer for hyperspectral band selection10.1016/j.asoc.2015.09.045 ↗
44. A bumble bees mating optimization algorithm for the feature selection problem
45. Hu, B., Dai, Y., Su, Y., Moore, P., Zhang, X., Mao, C., Chen, J., and Xu, L. (2016). Feature Selection for Optimized High-dimensional Biomedical Data using the Improved Shuffled Frog Leaping Algorithm. IEEE/ACM Trans. Comput. Biol. Bioinform.
46. Kernel-based learning and feature selection analysis for cancer diagnosis10.1016/j.asoc.2016.12.010 ↗
47. Enache, A.C., Sgarciu, V., and Petrescu-Nita, A. (2015, January 21–23). Intelligent feature selection method rooted in Binary Bat Algorithm for intrusion detection. Proceedings of the 10th Jubilee IEEE International Symposium on Applied Computational Intelligence and Informatics, Proceedings (SACI 2015), Timisoara, Romania.10.1109/SACI.2015.7208259 ↗
48. Packianather, M.S., and Kapoor, B. (2015, January 17–20). A wrapper-based feature selection approach using Bees Algorithm for a wood defect classification system. Proceedings of the 10th System of Systems Engineering Conference, SoSE 2015, San Antonio, TX, USA.10.1109/SYSOSE.2015.7151902 ↗
49. Neural-Discrete Hungry Roach Infestation Optimization to Select Informative Textural Features for Determining Water Content of Cultured Sunagoke Moss10.2525/ecb.49.1 ↗
50. Glowworm Swarm Based Informative Attribute Selection Using Support Vector Machines for Simultaneous Feature Selection and Classification10.1007/978-3-319-20294-5_3 ↗
51. Lin, K.C., Chen, S.Y., and Hung, J.C. (2015). Feature Selection for Support Vector Machines Base on Modified Artificial Fish Swarm Algorithm. Ubiquitous Computing Application and Wireless Sensor, Lecture Notes in Electrical Engineering 331, Springer.10.1007/978-94-017-9618-7_28 ↗
52. Feature Selection and Parameter Optimization of Support Vector Machines Based on Modified Artificial Fish Swarm Algorithms10.1155/2015/604108 ↗
53. An improved dynamic discrete firefly algorithm for blind image steganalysis10.1007/s13042-016-0610-3 ↗
54. A binary ABC algorithm based on advanced similarity scheme for feature selection10.1016/j.asoc.2015.07.023 ↗
55. Nakamura, R.Y.M., Pereira, L.A.M., Costa, K.A., Rodrigues, D., Papa, J.P., and Yang, X.S. (2012, January 22–25). BBA: A binary bat algorithm for feature selection. Proceedings of the Brazilian Symposium of Computer Graphic and Image Processing, Ouro Preto, Brazil.10.1109/SIBGRAPI.2012.47 ↗
56. Firefly as a novel swarm intelligence variable selection method in spectroscopy10.1016/j.aca.2014.09.045 ↗
57. Ant Colony Optimization Based Feature Selection Method for QEEG Data Classification10.4306/pi.2014.11.3.243 ↗
58. A highly accurate firefly based algorithm for heart disease prediction10.1016/j.eswa.2015.06.024 ↗
59. Jensen, R., Jensen, R., and Shen, Q. (2003, January 9–11). Finding Rough Set Reducts with Ant Colony Optimization. Proceedings of the 2003 UK Workshop on Computational Intelligence, Guilford, UK.
60. Modified binary particle swarm optimization10.1016/j.pnsc.2008.03.018 ↗
61. Khanesar, M.A., Teshnehlab, M., and Shoorehdeli, M.A. (2007, January 27–29). A novel binary particle swarm optimization. Proceedings of the IEEE 2007 Mediterranean Conference on Control & Automation, Athens, Greece.
62. A new hybrid ant colony optimization algorithm for feature selection10.1016/j.eswa.2011.09.073 ↗
63. Corchado, E., Wu, X., Oja, E., Herrero, Á., and Baruque, B. (2009). A Hybrid Bumble Bees Mating Optimization—GRASP Algorithm for Clustering. Hybrid Artificial Intelligence Systems, Proceedings of the 4th International Conference, HAIS 2009, Salamanca, Spain, 10–12 June 2009, Springer.10.1007/978-3-642-02319-4 ↗
64. Wang, H., Jing, X., and Niu, B. (2016, January 24–29). Bacterial-inspired feature selection algorithm and its application in fault diagnosis of complex structures. Proceedings of the 2016 IEEE Congress on Evolutionary Computation (CEC), Vancouver, BC, Canada.10.1109/CEC.2016.7744272 ↗
65. Feature selection based on an improved cat swarm optimization algorithm for big data classification10.1007/s11227-016-1631-0 ↗
66. Dara, S., and Banka, H. (2014, January 10–11). A Binary PSO Feature Selection Algorithm for Gene Expression Data. Proceedings of the 2014 International Conference on Advances in Communication and Computing Technologies, Mumbai, India.10.1109/EIC.2015.7230734 ↗
67. Hybrid Whale Optimization Algorithm with simulated annealing for feature selection10.1016/j.neucom.2017.04.053 ↗
68. A binary krill herd approach for feature selection
69. Dheeru, D., and Karra Taniskidou, E. (2018, June 20). UCI Machine Learning Repository. Available online: http://archive.ics.uci.edu/ml.
70. GEMS: A system for automated cancer diagnosis and biomarker discovery from microarray gene expression data10.1016/j.ijmedinf.2005.05.002 ↗
71. LIBSVM: A library for support vector machines10.1145/1961189.1961199 ↗
72. Li, J. (2018, June 20). Kent Ridge Bio-medical Dataset. Available online: http://leo.ugr.es/elvira/DBCRepository/.
73. A Rough Set Approach to Feature Selection Based on Wasp Swarm Optimization
74. Chen, K.H., Wang, K.J., Tsai, M.L., Wang, K.M., Adrian, A.M., Cheng, W.C., Yang, T.S., Teng, N.C., Tan, K.P., and Chang, K.S. (2014). Gene selection for cancer identification: A decision tree model empowered by particle swarm optimization algorithm. BMC Bioinform., 15.10.1186/1471-2105-15-49 ↗
75. Gene selection from microarray gene expression data for classification of cancer subgroups employing PSO and adaptive K-nearest neighborhood technique10.1016/j.eswa.2014.08.014 ↗
76. Pashaei, E., Ozen, M., and Aydin, N. (2015, January 25–29). Improving medical diagnosis reliability using Boosted C5.0 decision tree empowered by Particle Swarm Optimization. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.10.1109/EMBC.2015.7320060 ↗
77. Fong, S., Wong, R., and Vasilakos, A. (2015). Accelerated PSO Swarm Search Feature Selection for Data Stream Mining Big Data. IEEE Trans. Serv. Comput., 33–45.10.1109/TSC.2015.2439695 ↗
78. Tran, B., Xue, B., and Zhang, M. (2017). A New Representation in PSO for Discretization-Based Feature Selection. IEEE Trans. Cybern.10.1109/TCYB.2017.2714145 ↗
79. Finding rough set reducts with fish swarm algorithm10.1016/j.knosys.2015.02.002 ↗
80. Shahana, A.H., and Preeja, V. (2016, January 26–27). A binary krill herd approach based feature selection for high dimensional data. Proceedings of the IEEE 2016 International Conference on Inventive Computation Technologies (ICICT), Coimbatore, India.10.1109/INVENTIVE.2016.7830079 ↗
81. Hybrid Firefly Swarm Intelligence Based Feature Selection for Medical Data Classification and Segmentation in SVD—NSCT Domain10.21474/IJAR01/1544 ↗
82. A feature selection method based on modified binary coded ant colony optimization algorithm10.1016/j.asoc.2016.08.011 ↗
83. An ant colony optimization based feature selection for web page classification
84. Pal, M., Bhattacharyya, S., Roy, S., Konar, A., Tibarewala, D., and Janarthanan, R. (2014, January 22–25). A bacterial foraging optimization and learning automata based feature selection for motor imagery EEG classification. Proceedings of the IEEE 2014 International Conference on Signal Processing and Communications (SPCOM), Bangalore, India.10.1109/SPCOM.2014.6983926 ↗
85. Zhou, D., Fang, Y., Botzheim, J., Kubota, N., and Liu, H. (2016, January 6–9). Bacterial Memetic Algorithm based Feature Selection for Surface EMG based Hand Motion Recognition in Long-term Use. Proceedings of the 2016 IEEE Symposium Series on Computational Intelligence, Athens, Greece.10.1109/SSCI.2016.7850241 ↗
86. Wang, M., Wang, X., and Li, G. A improved speech synthesis system utilizing BPSO-based lip feature selection. Proceedings of the IEEE 2011 4th International Conference on Biomedical Engineering and Informatics (BMEI), Shanghai, China.10.1109/BMEI.2011.6098551 ↗
87. PSO-based automatic relevance determination and feature selection system for hyperspectral image classification10.1049/el.2012.0539 ↗
88. Hu, Z., Chiong, R., Pranata, I., Susilo, W., and Bao, Y. (2016, January 24–29). Identifying malicious web domains using machine learning techniques with online credibility and performance data. Proceedings of the 2016 IEEE Congress on Evolutionary Computation (CEC), Vancouver, BC, Canada.10.1109/CEC.2016.7748347 ↗
89. Sattiraju, M., Manikandan, M.V., Manikantan, K., and Ramachandran, S. (2013, January 18–21). Adaptive BPSO based feature selection and skin detection based background removal for enhanced face recognition. Proceedings of the 2013 IEEE Fourth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG), Jodhpur, India.10.1109/NCVPRIPG.2013.6776226 ↗
90. Dehuri, S., Roy, R., and Cho, S.B. (2011, January 12–16). An adaptive binary PSO to learn bayesian classifier for prognostic modeling of metabolic syndrome. Proceedings of the 13th annual conference companion on Genetic and evolutionary computation (GECCO ’11), Dublin, Ireland.10.1145/2001858.2002039 ↗
91. Applying particle swarm optimization-based decision tree classifier for cancer classification on gene expression data10.1016/j.asoc.2014.08.032 ↗
92. An Enhanced Grey Wolf Optimization Based Feature Selection Wrapped Kernel Extreme Learning Machine for Medical Diagnosis10.1155/2017/9512741 ↗
93. Gene Selection from Microarray Data Using Binary Grey Wolf Algorithm for Classifying Acute Leukemia10.2174/1574362411666160607084415 ↗
94. Seth, J.K., and Chandra, S. (2016, January 16–18). Intrusion detection based on key feature selection using binary GWO. Proceedings of the IEEE 3rd International Conference on Computing for Sustainable Global Development (INDIACom), New Delhi, India.
95. Sargo, J.A.G., Vieira, S.M., Sousa, J.M.C., and Filho, C.J.B. (2014, January 6–11). Binary Fish School Search applied to feature selection: Application to ICU readmissions. Proceedings of the 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Beijing, China.10.1109/FUZZ-IEEE.2014.6891802 ↗
96. Network Intrusion Detection Based on the Improved Artificial Fish Swarm Algorithm10.4304/jcp.8.11.2990-2996 ↗
97. Dai, Y., Hu, B., Su, Y., Mao, C., Chen, J., Zhang, X., Moore, P., Xu, L., and Cai, H. (2015, January 9–12). Feature selection of high-dimensional biomedical data using improved SFLA for disease diagnosis. Proceedings of the 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Washington, DC, USA.
98. Ladgham, A., Torkhani, G., Sakly, A., and Mtibaa, A. (2013, January 6–8). Modified support vector machines for MR brain images recognition. Proceedings of the 2013 IEEE International Conference on Control, Decision and Information Technologies (CoDIT), Hammamet, Tunisia.10.1109/CoDIT.2013.6689515 ↗
99. Weight Optimized Neural Network Using Metaheuristics for the Classification of Large Cell Carcinoma and Adenocarcinoma from Lung Imaging10.2174/1574362411666160622095146 ↗
100. Wang, L., Jia, P., Huang, T., Duan, S., Yan, J., and Wang, L. (2016). A Novel Optimization Technique to Improve Gas Recognition by Electronic Noses Based on the Enhanced Krill Herd Algorithm. Sensors, 16.10.3390/s16081275 ↗