Comparison of Selection Criteria for Model Selection of Support Vector Machine on Physiological Data with Inter-Subject Variance
Source abstract
Abstract
Support vector machines (SVMs) utilize hyper-parameters for classification. Model selection (MS) is an essential step in the construction of the SVM classifier as it involves the identification of the appropriate parameters. Several selection criteria have been proposed for MS, but their usefulness is limited for physiological data exhibiting inter-subject variance (ISV) that makes different characteristics between training and test data. To identify an effective solution for the constraint, this study considered a leave-one-subject-out cross validation-based selection criterion (LSSC) with six well-known selection criteria and compared their effectiveness. Nine classification problems were examined for the comparison, and the MS results of each selection criterion were obtained and analyzed. The results showed that the SVM model selected by the LSSC yielded the highest average classification accuracy among all selection criteria in the nine problems. The average accuracy was 2.96% higher than that obtained with the conventional K-fold cross validation-based selection criterion. In addition, the advantage of the LSSC was more evident for data with larger ISV. Thus, the results of this study can help optimize SVM classifiers for physiological data and are expected to be useful for the analysis of physiological data to develop various medical decision systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: ECG Monitoring and Analysis · EEG and Brain-Computer Interfaces · Heart Rate Variability and Autonomic Control
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Creative IT Engineering, Pohang University of Science and Technology, Jigok-ro 80, Pohang 37673, Korea
School of Electronic and Electrical Engineering, Daegu Catholic University, Hayang-ro 13-13, Gyeongsan 38430, Korea
References (45)
Show source-provided references
1. QRS detection method based on fully convolutional networks for capacitive electrocardiogram10.1016/j.eswa.2019.05.033 ↗
2. Lynn, H.M., Kim, P., and Pan, S.B. (2021). Data Independent Acquisition Based Bi-Directional Deep Networks for Biometric ECG Authentication. Appl. Sci., 11.10.3390/app11031125 ↗
3. Tirado-Martin, P., and Sanchez-Reillo, R. (2021). BioECG: Improving ECG Biometrics with Deep Learning and Enhanced Datasets. Appl. Sci., 11.10.3390/app11135880 ↗
4. Zhang, P., Li, F., Zhao, R., Zhou, R., Du, L., Zhao, Z., Chen, X., and Fang, Z. (2021). Real-Time Psychological Stress Detection According to ECG Using Deep Learning. Appl. Sci., 11.10.3390/app11093838 ↗
5. Ayoobi, N., Sharifrazi, D., Alizadehsani, R., Shoeibi, A., Gorriz, J.M., Moosaei, H., Khosravi, A., Nahavandi, S., Chofreh, A.G., and Goni, F.A. (2021). Time Series Forecasting of New Cases and New Deaths Rate for COVID-19 using Deep Learning Methods. arXiv.10.1016/j.rinp.2021.104495 ↗
6. Generalized Twin Support Vector Machines10.1007/s11063-021-10464-3 ↗
7. Mangasarian, O.L. (2001, January 23–27). Data mining via support vector machines. Proceedings of the IFIP Conference on System Modeling and Optimization, Trier, Germany.
8. SSVM: A smooth support vector machine for classification10.1023/A:1011215321374 ↗
9. Wearable Device-Based System to Monitor a Driver’s Stress, Fatigue, and Drowsiness10.1109/TIM.2017.2779329 ↗
10. Ortega, S., Fabelo, H., Halicek, M., Camacho, R., Plaza, M.d.l.L., Callicó, G.M., and Fei, B. (2020). Hyperspectral superpixel-wise glioblastoma tumor detection in histological samples. Appl. Sci., 10.10.3390/app10134448 ↗
11. Setiowati, S., Franita, E.L., and Ardiyanto, I. (2017, January 12–13). A review of optimization method in face recognition: Comparison deep learning and non-deep learning methods. Proceedings of the 9th International Conference on Information Technology and Electrical Engineering (ICITEE), Phuket, Thailand.10.1109/ICITEED.2017.8250484 ↗
12. Pandit, R., and Kolios, A. (2020). SCADA data-based support vector machine wind turbine power curve uncertainty estimation and its comparative studies. Appl. Sci., 10.10.3390/app10238685 ↗
13. Rizwan, A., Iqbal, N., Ahmad, R., and Kim, D.H. (2021). WR-SVM Model Based on the Margin Radius Approach for Solving the Minimum Enclosing Ball Problem in Support Vector Machine Classification. Appl. Sci., 11.10.3390/app11104657 ↗
14. Automatic model selection for the optimization of SVM kernels10.1016/j.patcog.2005.03.011 ↗
15. Optimizing resources in model selection for support vector machine10.1016/j.patcog.2006.06.012 ↗
16. Support vector machine with parameter optimization by a novel hybrid method and its application to fault diagnosis10.1016/j.neucom.2014.08.010 ↗
17. A dynamic model selection strategy for support vector machine classifiers10.1016/j.asoc.2012.04.001 ↗
18. Multi-objective uniform design as a SVM model selection tool for face recognition10.1016/j.eswa.2010.11.066 ↗
19. Model selection for support vector machines via uniform design10.1016/j.csda.2007.02.013 ↗
20. A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy10.1016/j.eswa.2005.12.008 ↗
21. Namdeo, A., and Singh, D. (2021). Challenges in evolutionary algorithm to find optimal parameters of SVM: A review. Mater. Today-Proc.10.1016/j.matpr.2021.03.288 ↗
22. Bounds on error expectation for support vector machines10.1162/089976600300015042 ↗
23. Hyperparameter design criteria for support vector classifiers10.1016/S0925-2312(03)00430-2 ↗
24. Analysis of the distance between two classes for tuning SVM hyperparameters10.1109/TNN.2009.2036999 ↗
25. Tuning kernel parameters for SVM based on expected square distance ratio10.1016/j.ins.2016.07.047 ↗
26. Evaluation of simple performance measures for tuning SVM hyperparameters10.1016/S0925-2312(02)00601-X ↗
27. Empirical comparison of cross-validation and internal metrics for tuning SVM hyperparameters10.1016/j.patrec.2017.01.007 ↗
28. Choi, M., Seo, M., Lee, J.S., and Kim, S.W. (2020). Fuzzy support vector machine-based personalizing method to address the inter-subject variance problem of physiological signals in a driver monitoring system. Artif. Intell. Med., 105.10.1016/j.artmed.2020.101843 ↗
29. Deep Neural Networks for Human Activity Recognition with Wearable Sensors: Leave-one-subject-out Cross-validation for Model Selection10.1109/ACCESS.2020.3010715 ↗
30. Optimal hyper-parameter tuning of SVM classifiers with application to medical diagnosis
31. Kumar, S. (2004). Neural Networks: A Classroom Approach, Tata McGraw-Hill Education.
32. Improving classification performance of support vector machine by genetically optimising kernel shape and hyper-parameters10.1007/s10489-010-0260-1 ↗
33. Detecting stress during real-world driving tasks using physiological sensors10.1109/TITS.2005.848368 ↗
34. Physiobank, physiotoolkit, and physionet10.1161/01.CIR.101.23.e215 ↗
35. Koldijk, S., Sappelli, M., Verberne, S., Neerincx, M.A., and Kraaij, W. (2014, January 12–16). The swell knowledge work dataset for stress and user modeling research. Proceedings of the 16th International Conference on Multimodal Interaction, Istanbul, Turkey.10.1145/2663204.2663257 ↗
36. Nkurikiyeyezu, K., Yokokubo, A., and Lopez, G. (2019). The Effect of Person-Specific Biometrics in Improving Generic Stress Predictive Models. arXiv.10.18494/SAM.2020.2650 ↗
37. Leeb, R., Brunner, C., Müller-Putz, G., Schlögl, A., and Pfurtscheller, G. (2008). BCI Competition 2008–Graz Data Set B, Graz University of Technology.
38. Lopez, G., Kawahara, Y., Suzuki, Y., Takahashi, M., Takahashi, H., and Wada, M. (2016). Effect of direct neck cooling on psychological and physiological state in summer heat environment. Mech. Eng. J., 3.10.1299/mej.15-00537 ↗
39. The impact of the MIT-BIH arrhythmia database10.1109/51.932724 ↗
40. Heartbeat classification fusing temporal and morphological information of ECGs via ensemble of classifiers10.1016/j.bspc.2018.08.007 ↗
41. Ang, K.K., Chin, Z.Y., Wang, C., Guan, C., and Zhang, H. (2012). Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b. Front. Neurosci., 6.10.3389/fnins.2012.00039 ↗
42. Nkurikiyeyezu, K., Yokokubo, A., and Lopez, G. (2019, January 3–6). Affect-aware thermal comfort provision in intelligent buildings. Proceedings of the 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), Cambridge, UK.10.1109/ACIIW.2019.8925184 ↗
43. Son, Y., and Kim, W. (2020). Missing Value Imputation in Stature Estimation by Learning Algorithms Using Anthropometric Data: A Comparative Study. Appl. Sci., 10.10.3390/app10145020 ↗
44. Le, N.Q.K., Hung, T.N.K., Do, D.T., Lam, L.H.T., Dang, L.H., and Huynh, T.T. (2021). Radiomics-based machine learning model for efficiently classifying transcriptome subtypes in glioblastoma patients from MRI. Comput. Biol. Med., 132.10.1016/j.compbiomed.2021.104320 ↗
45. Using extreme gradient boosting to identify origin of replication in Saccharomyces cerevisiae via hybrid features10.1016/j.ygeno.2020.01.017 ↗