journal article·2025-07-23Open access

Leveraging Feature Extraction to Perform Time-Efficient Selection for Machine Learning Applications

Duarte Coelho, Ana Madureira, Ivo Pereira, Ramiro Gonçalves, Susana Nicola, Inês César, Daniel Alves de Oliveira

Applied Sciences· Volume 15· Issue 15

Abstract

In the age of rapidly advancing machine learning capabilities, the pursuit of maximum performance encounters the practical limitations imposed by limited resources in several fields. This work presents a cost-effective proposal for feature selection, which is a crucial part of machine learning processes, and intends to partly solve this problem through computational time reduction. The proposed methodology aims to strike a careful balance between feature exploration and strict computational time concerns, by enhancing the quality and relevance of data. This approach focuses on the use of interim representations of feature combinations to significantly speed up a potentially slow and computationally expensive process. This strategy is evaluated in several datasets against other feature selection methods, and the results indicate a significant reduction in the temporal costs associated with this process, achieving a mean percentage decrease of 85%. Furthermore, this reduction is achieved while maintaining competitive model performance, demonstrating that the selected features remain effective for the learning task. These results emphasize the method’s feasibility, confirming its ability to transform machine learning applications in environments with limited resources.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Neural Networks and Applications · Metaheuristic Optimization Algorithms Research · Face and Expression Recognition

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

ISRC, ISEP, Polytechnic of Porto, Rua Dr. António Bernardino de Almeida, 4249-015 Porto, Portugal

Department of Engineering, School of Sciences and Technology, Universidade de Trás-os-Montes e Alto Douro, 5000-801 Vila Real, Portugal

E-goi, 4450-190 Matosinhos, Portugal

INESC INOV, 1000-029 Lisboa, Portugal

Instituto de Engenharia de Sistemas e Computadores, Tecnologia e Ciência (INESC TEC), Faculdade de Engenharia da Universidade do Porto, Rua Dr. Roberto Frias, 4200-465 Porto, Portugal

Funding

FCT—Fundação para a Ciência e Tecnologia

References (100+)

Show source-provided references
1. A survey on data collection for machine learning: A big data-ai integration perspective10.1109/TKDE.2019.2946162 ↗
2. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. (2022). Hierarchical text-conditional image generation with clip latents. arXiv.
3. Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., and Anadkat, S. (2023). Gpt-4 technical report. arXiv.
4. Machine learning to detect signatures of disease in liquid biopsies–a user’s guide10.1039/C7LC00955K ↗
5. Esmeli, R., Bader-El-Den, M., and Abdullahi, H. (2020, January 2–4). Session similarity based approach for alleviating cold-start session problem in e-commerce for Top-N recommendations. Proceedings of the 12th International Joint Conference on Knowledge Discovery, Budapest, Hungary.10.5220/0010107001790186 ↗
6. Tabatabaei, S.A., Hoogendoorn, M., and van Halteren, A. (November, January 29). Narrowing reinforcement learning: Overcoming the cold start problem for personalized health interventions. Proceedings of the PRIMA 2018: Principles and Practice of Multi-Agent Systems: 21st International Conference, Tokyo, Japan.10.1007/978-3-030-03098-8_19 ↗
7. Qiao, S., Liu, C., Shen, W., and Yuille, A.L. (2018, January 18–23). Few-shot image recognition by predicting parameters from activations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00755 ↗
8. Emoji, playfulness, and brand engagement on twitter10.1016/j.intmar.2020.06.002 ↗
9. Phygital technologies and environments for breakthrough innovation in customers’ and citizens’ journey. A critical literature review and future agenda10.1016/j.techfore.2023.122342 ↗
10. A comprehensive survey on sentiment analysis: Approaches, challenges and trends10.1016/j.knosys.2021.107134 ↗
11. Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions10.1016/j.inffus.2022.09.025 ↗
12. Talbi, E.G. (2013). Hybrid Metaheuristics, Springer.10.1007/978-3-642-30671-6 ↗
13. Digital medicine and the curse of dimensionality10.1038/s41746-021-00521-5 ↗
14. Conceptual and empirical comparison of dimensionality reduction algorithms (PCA, KPCA, LDA, MDS, SVD, LLE, ISOMAP, LE, ICA, t-SNE)10.1016/j.cosrev.2021.100378 ↗
15. A comprehensive survey on feature selection in the various fields of machine learning10.1007/s10489-021-02550-9 ↗
16. Feature selection in machine learning: A new perspective10.1016/j.neucom.2017.11.077 ↗
17. Pudjihartono, N., Fadason, T., Kempa-Liehr, A.W., and O’Sullivan, J.M. (2022). A review of feature selection methods for machine learning-based disease risk prediction. Front. Bioinform., 2.10.3389/fbinf.2022.927312 ↗
18. A survey on feature selection10.1016/j.procs.2016.07.111 ↗
19. A comprehensive survey on recent metaheuristics for feature selection10.1016/j.neucom.2022.04.083 ↗
20. Binary optimization using hybrid grey wolf optimization for feature selection10.1109/ACCESS.2019.2906757 ↗
21. Mayfly in harmony: A new hybrid meta-heuristic feature selection algorithm10.1109/ACCESS.2020.3031718 ↗
22. Ho, V.N., and Vovk, S.-S. (2014). Feature selection. Balasubramanian, Conformal Prediction for Reliable Machine Learning: Theory, Adaptations and Applications, Elsevier.
23. Verhaeghe, J., Donckt, J.V.D., Ongenae, F., and Hoecke, S.V. (2023). Powershap: A power-full shapley feature selection method. Lecture Notes in Computer Science, Springer.10.1007/978-3-031-26387-3_5 ↗
24. Wrapper and hybrid feature selection methods using metaheuristic algorithms for english text classification: A systematic review10.1109/ACCESS.2022.3165814 ↗
25. Hamla, H., and Ghanem, K. (2021, January 25–27). Comparative study of embedded feature selection methods on microarray data. Proceedings of the Artificial Intelligence Applications and Innovations: 17th IFIP WG 12.5 International Conference, AIAI 2021, Hersonissos, Crete, Greece.
26. Building Highly Reliable Quantitative Structure–Activity Relationship Classification Models Using the Rivality Index Neighborhood Algorithm with Feature Selection10.1021/acs.jcim.9b00706 ↗
27. Dimensionality reduction: A comparative review
28. Survey of feature selection and extraction techniques for stock market prediction10.1186/s40854-022-00441-7 ↗
29. Feature dimensionality reduction: A review10.1007/s40747-021-00637-x ↗
30. Abraham, A., Bajaj, A., Gandhi, N., Madureira, A.M., and Kahraman, C. (2023). A Review on Dimensionality Reduction for Machine Learning. Innovations in Bio-Inspired Computing and Applications, Springer.10.1007/978-3-031-27499-2 ↗
31. Glover, F.W., and Kochenberger, G.A. (2006). Handbook of Metaheuristics, Springer Science & Business Media.
32. Metaheuristics in combinatorial optimization: Overview and conceptual comparison10.1145/937503.937505 ↗
33. Abraham, A., Madureira, A.M., Kaklauskas, A., Gandhi, N., Bajaj, A., Muda, A.K., Kriksciuniene, D., and Ferreira, J.C. (2022). A Review on MOEA and Metaheuristics for Feature-Selection. Innovations in Bio-Inspired Computing and Applications, Springer.10.1007/978-3-030-96299-9 ↗
34. Binary bat algorithm10.1007/s00521-013-1525-5 ↗
35. Feature selection using different transfer functions for binary bat algorithm
36. Feature Selection for Image Steganalysis Using Binary Bat Algorithm10.1109/ACCESS.2019.2963084 ↗
37. Nakamura, R.Y., Pereira, L.A., Costa, K.A., Rodrigues, D., Papa, J.P., and Yang, X.S. (2012, January 22–25). BBA: A binary bat algorithm for feature selection. Proceedings of the 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images, Ouro Preto, Brazil.10.1109/SIBGRAPI.2012.47 ↗
38. Rodrigues, D., Pereira, L.A., Almeida, T., Papa, J.P., Souza, A., Ramos, C.C., and Yang, X.S. (2013, January 19–23). BCS: A binary cuckoo search algorithm for feature selection. Proceedings of the 2013 IEEE International Symposium on Circuits and Systems (ISCAS), Beijing, China.10.1109/ISCAS.2013.6571881 ↗
39. Yang, X.S., and Deb, S. (2010). Cuckoo Search via Levy Flights. arXiv.10.1109/NABIC.2009.5393690 ↗
40. Memory based cuckoo search algorithm for feature selection of gene expression dataset10.1016/j.imu.2021.100572 ↗
41. Yang, X.S. (2014). Chapter 9—Cuckoo Search. Nature-Inspired Optimization Algorithms, Elsevier.10.1016/B978-0-12-416743-8.00009-9 ↗
42. Enhanced Binary Cuckoo Search with Frequent Values and Rough Set Theory for Feature Selection10.1109/ACCESS.2021.3107901 ↗
43. Equilibrium optimizer: A novel optimization algorithm10.1016/j.knosys.2019.105190 ↗
44. An efficient binary equilibrium optimizer algorithm for feature selection10.1109/ACCESS.2020.3013617 ↗
45. An improved equilibrium optimizer algorithm for features selection: Methods and analysis10.1109/ACCESS.2021.3108097 ↗
46. Sampson, J.R. (2025, February 01). Adaptation in Natural and Artificial Systems (John H. Holland). Available online: https://epubs.siam.org/doi/10.1137/1018105.
47. Lambora, A., Gupta, K., and Chopra, K. (2019, January 14–16). Genetic algorithm-A literature review. Proceedings of the 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), Faridabad, India.10.1109/COMITCon.2019.8862255 ↗
48. Mathew, T.V. (2012). Genetic algorithm. Rep. Submitt. IIT Bombay, 53, Available online: https://datajobs.com/data-science-repo/Genetic-Algorithm-Guide-[Tom-Mathew].pdf.
49. An evolutionary gravitational search-based feature selection10.1016/j.ins.2019.05.038 ↗
50. GSA: A gravitational search algorithm10.1016/j.ins.2009.03.004 ↗
51. Halliday, D., Resnick, R., Walker, J., Halliday, D., Resnick, R., and Walker, J. (2000). Extended, Fundamentals of Physics, John Wiley and Sons.
52. Papa, J.P., Pagnin, A., Schellini, S.A., Spadotto, A., Guido, R.C., Ponti, M., Chiachia, G., and Falcão, A.X. (2011, January 22–27). Feature selection through gravitational search algorithm. Proceedings of the 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic.10.1109/ICASSP.2011.5946916 ↗
53. A New Heuristic Optimization Algorithm: Harmony Search10.1177/003754970107600201 ↗
54. A novel algorithm for feature selection using harmony search and its application for non-technical losses detection10.1016/j.compeleceng.2011.09.013 ↗
55. Feature selection with harmony search10.1109/TSMCB.2012.2193613 ↗
56. Feature selection for high dimensional imbalanced class data using harmony search10.1016/j.engappai.2016.10.008 ↗
57. A review of Harmony Search algorithm-based feature selection method for classification10.1088/1742-6596/1192/1/012038 ↗
58. A mayfly optimization algorithm10.1016/j.cie.2020.106559 ↗
59. Fard, A., and Hajiaghaei-Keshteli, M. (2016, January 25–26). Red Deer Algorithm (RDA); a new optimization algorithm inspired by Red Deers’ mating. Proceedings of the International Conference on Industrial Engineering, Tehran, Iran.
60. Red deer algorithm (RDA): A new nature-inspired meta-heuristic10.1007/s00500-020-04812-z ↗
61. Review and analysis for the Red Deer Algorithm10.1007/s12652-021-03602-1 ↗
62. The whale optimization algorithm10.1016/j.advengsoft.2016.01.008 ↗
63. A comprehensive survey: Whale Optimization Algorithm and its applications10.1016/j.swevo.2019.03.004 ↗
64. Whale optimization algorithm: A systematic review of contemporary applications, modifications and developments10.1007/s00521-020-04849-z ↗
65. Volkovs, M., Yu, G., and Poutanen, T. (2017). Dropoutnet: Addressing cold start in recommender systems. Adv. Neural Inf. Process. Syst., 30.
66. Cold start approach for data-driven fault detection10.1109/TII.2012.2231870 ↗
67. Berman, G., and Fryer, K. (2014). Introduction to Combinatorics, Elsevier Science.
68. Rosen, K. (2011). Loose Leaf for Discrete Mathematics and Its Applications, McGraw-Hill Education.
69. Guntur, R. (2024, October 01). API Security: Access Behavior Anomaly Dataset. Available online: https://www.kaggle.com/datasets/tangodelta/api-access-behaviour-anomaly-dataset.
70. Rashmi (2024, October 01). Banking Dataset Classification. Available online: https://www.kaggle.com/datasets/rashmiranu/banking-dataset-classification.
71. Moro, S., Rita, P., and Cortez, P. (2024, October 01). Bank Marketing. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/222/bank+marketing.
72. Wolberg, W. (2024, October 01). Breast Cancer Wisconsin (Original). UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/15/breast+cancer+wisconsin+original.
73. Iyyer, S. (2024, October 01). Churn Modelling. Available online: https://www.kaggle.com/datasets/shrutimechlearn/churn-modelling.
74. Taiwanese Bankruptcy Prediction (2024, October 01). UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/572/taiwanese+bankruptcy+prediction.
75. Lillelund, C. (2024, October 01). CS:GO Round Winner Classification. Available online: https://www.kaggle.com/datasets/christianlillelund/csgo-round-winner-classification.
76. Unmoved (2024, October 01). Cure The Princess. Available online: https://www.kaggle.com/datasets/unmoved/cure-the-princess.
77. Babativa, D. (2024, October 01). Depression. Available online: https://www.kaggle.com/datasets/diegobabativa/depression.
78. Roesler, O. (2024, October 01). EEG Eye State. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/264/eeg+eye+state.
79. Hofmann, H. (2024, October 01). Statlog (German Credit Data). UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/144/statlog+german+credit+data.
80. Hothorn, T., and Lausen, B. (2024, October 01). GlaucomaM. Available online: https://r-packages.io/datasets/GlaucomaM.
81. Fedesoriano (2024, October 01). Heart Failure Prediction Dataset. Available online: https://www.kaggle.com/datasets/fedesoriano/heart-failure-prediction.
82. Konapure, R. (2024, October 01). Home Loan Approval. Available online: https://www.kaggle.com/datasets/rishikeshkonapure/home-loan-approval.
83. Yasser H, M. (2024, October 01). IBM Attrition Dataset. Available online: https://www.kaggle.com/datasets/yasserh/ibm-attrition-dataset.
84. Sigillito, V., Wing, S., Hutton, L., and Baker, K. (2024, October 01). Ionosphere. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/52/ionosphere.
85. Bock, R. (2024, October 01). MAGIC Gamma Telescope. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/159/magic+gamma+telescope.
86. Rastog, S. (2024, October 01). Microcalcification Classification. Available online: https://www.kaggle.com/datasets/sudhanshu2198/microcalcification-classification.
87. Chapman, D., and Jain, A. (2024, October 01). Musk (Version 1). UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/74/musk+version+1.
88. Little, M. (2024, October 01). Parkinsons. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/174/parkinsons.
89. Sigillito, V. (2024, October 01). Pima Indians Diabetes Database. UCI Machine Learning Repository. Available online: https://www.kaggle.com/datasets/uciml/pima-indians-diabetes-database.
90. Lyon, R. (2024, October 01). HTRU2. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/372/htru2.
91. MsSmartyPants (2024, October 01). Rice Type Classification. Available online: https://www.kaggle.com/datasets/mssmartypants/rice-type-classification.
92. Blattmann, S. (2024, October 01). Smoke Detection Dataset. Available online: https://www.kaggle.com/datasets/deepcontractor/smoke-detection-dataset.
93. Sejnowski, T., and Gorman, R. (2024, October 01). Connectionist Bench (Sonar, Mines vs. Rocks). UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/151/connectionist+bench+sonar+mines+vs+rocks.
94. Hopkins, M., Reeber, E., Forman, G., and Suermondt, J. (2024, October 01). Spambase. UCI Machine Learning Repository. Available online: https://archive.ics.uci.edu/dataset/94/spambase.
95. McIntire, G. (2024, October 01). Spotify Song Attributes. Available online: https://www.kaggle.com/datasets/geomack/spotifyclassification.
96. Dincer, B. (2024, October 01). Telescope Spectrum Gamma or Hadron. Available online: https://www.kaggle.com/datasets/brsdincer/telescope-spectrum-gamma-or-hadron/data.
97. Frank, E., and Harrell, T.C. (2024, October 01). Titanic Dataset. Vanderbilt Biostatistics. Available online: https://hbiostat.org/data/repo/titanic.
98. MsSmartyPants (2024, October 01). Water Quality. Available online: https://www.kaggle.com/datasets/mssmartypants/water-quality.
99. Mehta, A. (2024, October 01). Personality Classification Data: 16 Personalities. Available online: https://www.kaggle.com/datasets/anshulmehtakaggl/60k-responses-of-16-personalities-test-mbt.
100. Kuila, A. (2024, October 01). Customer Segmentation Classification. Available online: https://www.kaggle.com/datasets/akashdeepkuila/automobile-customer.