Mental Workload Assessment Using Machine Learning Techniques Based on EEG and Eye Tracking Data
Source abstract
Abstract
The main contribution of this study was the concurrent application of EEG and eye tracking techniques during n-back tasks as part of the methodology for addressing the problem of mental workload classification through machine learning algorithms. The experiments involved 15 university students, consisting of 7 women and 8 men. Throughout the experiments, the researchers utilized the n-back memory task and the NASA-Task Load Index (TLX) subjective rating scale to assess various levels of mental workload. The results indicating the relationship between EEG and eye tracking measures and mental workload are consistent with previous research. Regarding the four-class classification task, mental workload level could be predicted with 76.59% accuracy using 34 selected features. This study makes a significant contribution to the literature by presenting a four-class mental workload estimation model that utilizes different machine learning algorithms.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human-Automation Interaction and Safety · EEG and Brain-Computer Interfaces · Healthcare Technology and Patient Monitoring
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
The Scientific and Technological Research Council of Türkiye, Ankara 06530, Türkiye
Department of Industrial Engineering, Gazi University, Ankara 06570, Türkiye
Funding
Scientific Research Projects Unit at Gazi University
References (68)
Show source-provided references
1. Hancock, P.A., and Desmond, P.A. (2001). Stress, Workload, and Fatigue, Lawrence Erlbau.10.1201/b12791 ↗
2. Cognitive load during problem solving: Effects on learning10.1207/s15516709cog1202_4 ↗
3. A theoretical framework for evaluating mental workload resources in human systems design for manufacturing operations10.1016/j.ergon.2016.10.007 ↗
4. Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA-TLX, and Workload Profile Methods10.1111/j.1464-0597.2004.00161.x ↗
5. Using eye movement activity as a correlate of cognitive workload10.1016/j.ergon.2006.04.002 ↗
6. Driver workload and eye blink duration10.1016/j.trf.2010.12.001 ↗
7. Mental workload measurement for emergency operating procedures in digital nuclear power plants10.1080/00140139.2013.790483 ↗
8. Estimating mental workload through event-related fluctuations of pupil area during a task in a virtual world10.1016/j.ijpsycho.2013.11.002 ↗
9. Influence of mental workload on detecting information varieties revealed by mismatch negativity during flight simulation10.1016/j.ergon.2017.08.004 ↗
10. Using pupillometry and gaze-based metrics for understanding drivers’ mental workload during automated driving10.1016/j.trf.2023.02.015 ↗
11. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research10.1016/S0166-4115(08)62386-9 ↗
12. Grimes, D., Tan, D.S., Hudson, S.E., Shenoy, P., and Rao, R.P. (2008, January 5–10). Feasibility and pragmatics of classifying working memory load with an electroencephalograph. Proceedings of the CHI ′08: CHI Conference on Human Factors in Computing Systems, Florence, Italy.10.1145/1357054.1357187 ↗
13. Multisubject “Learning” for Mental Workload Classification Using Concurrent EEG, fNIRS, and Physiological Measures10.3389/fnhum.2017.00389 ↗
14. Jusas, V., and Samuvel, S.G. (2019). Classification of Motor Imagery Using a Combination of User-Specific Band and Subject-Specific Band for Brain-Computer Interface. Appl. Sci., 9.10.3390/app9234990 ↗
15. Cross-session classification of mental workload levels using EEG and an adaptive deep learning model10.1016/j.bspc.2016.11.013 ↗
16. EEG-Based Multiclass Workload Identification Using Feature Fusion and Selection
17. Cross-subject workload classification with a hierarchical Bayes model10.1016/j.neuroimage.2011.07.094 ↗
18. Kaczorowska, M., Plechawska-Wójcik, M., and Tokovarov, M. (2021). Interpretable Machine Learning Models for Three-Way Classification of Cognitive Workload Levels for Eye-Tracking Features. Brain Sci., 11.10.3390/brainsci11020210 ↗
19. Discrimination of Mental Workload Levels in Human Subjects with Functional Near-Infrared Spectroscopy10.1142/S1793545808000224 ↗
20. Using Artificial Neural Networks for Predicting Mental Workload in Nuclear Power Plants Based on Eye Tracking10.1080/00295450.2019.1620055 ↗
21. Automatic recognition of alertness level from EEG by using neural network and wavelet coefficients10.1016/j.eswa.2004.12.027 ↗
22. Recognising situation awareness associated with different workloads using EEG and eye-tracking features in air traffic control tasks10.1016/j.knosys.2022.110179 ↗
23. Borys, M., Plechawska-Wójcik, M., Wawrzyk, M. (2017, January 12–14). Classifying cognitive workload using eye activity and EEG features in arithmetic tasks. Proceedings of the Information and Software Technologies: 23rd International Conference, ICIST 2017, Druskininkai, Lithuania.10.1007/978-3-319-67642-5_8 ↗
24. Kaczorowska, M., Wawrzyk, M., and Plechawska-Wójcik, M. (2020, January 16–18). Binary Classification of Cognitive Workload Levels with Oculog-raphy Features. Proceedings of the Computer Information Systems and Industrial Management: 19th International Conference, CISIM 2020, Bialystok, Poland. Proceedings 19.10.1007/978-3-030-47679-3_21 ↗
25. Qu, H., Shan, Y., Liu, Y., Pang, L., Fan, Z., Zhang, J., and Wanyan, X. (2020). Mental workload classification method based on EEG independent component features. Appl. Sci., 10.10.3390/app10093036 ↗
26. STEW: Simultaneous task EEG workload dataset10.1109/TNSRE.2018.2872924 ↗
27. EEG-based mental workload estimation of multiple sclerosis patients10.1007/s11760-023-02547-6 ↗
28. Cross-Task Cognitive Workload Recognition Based on EEG and Domain Adaptation10.1109/TNSRE.2022.3140456 ↗
29. Plechawska-Wójcik, M., Tokovarov, M., Kaczorowska, M., and Zapała, D. (2019). A Three-Class Classification of Cognitive Workload Based on EEG Spectral Data. Appl. Sci., 9.10.3390/app9245340 ↗
30. A Novel Method for Classifying Driver Mental Workload Under Naturalistic Conditions with Information from Near-Infrared Spectroscopy10.3389/fnhum.2018.00431 ↗
31. Mental workload during n-back task—Quantified in the prefrontal cortex using fNIRS10.3389/fnhum.2013.00935 ↗
32. Towards an effective cross-task mental workload recognition model using electroencephalography based on feature selection and support vector machine regression10.1016/j.ijpsycho.2015.10.004 ↗
33. Classification of a Driver’s cognitive workload levels using artificial neural network on ECG signals10.1016/j.apergo.2016.09.013 ↗
34. A machine learning approach to classify mental workload based on eye tracking data
35. Aksu, Ş.H., Çakıt, E., and Dağdeviren, M. (2023, January 22–24). Investigating the Relationship Between EEG Features and N-Back Task Difficulty Levels with NASA-TLX Scores Among Undergraduate Students. Proceedings of the Intelligent Human Systems Integration (IHSI 2023): Integrating People and Intelligent Systems, 69, New York, NY, USA.
36. Aksu, Ş.H., and Çakıt, E. (2022, January 24–28). Classifying mental workload using EEG data: A machine learning approach. Proceedings of the International Conference on Applied Human Factors and Ergonomics (AHFE), New York, NY, USA.
37. Oztürk, A. (2018). Transfer and Maintenance Effects of N-Back Working Memory Training in Interpreting Students: A Behavioural and Optical Brain Imaging Study. [Ph.D. Thesis, Middle East Technical University, Informatics Institute].
38. Monod, H., and Kapitaniak, B. (1999). Ergonomie, Masson Publishing.
39. What is the relationship between mental workload factors and cognitive load types?10.1016/j.ijpsycho.2011.09.023 ↗
40. Guan, K., Chai, X., Zhang, Z., Li, Q., and Niu, H. (2021, January 1–5). Evaluation of mental workload in working memory tasks with different information types based on EEG. Proceedings of the 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Virtual.10.1109/EMBC46164.2021.9630575 ↗
41. EEGLAB: An Open Source Toolbox for Analysis of Single-Trial EEG Dynamics Including Independent Component Analysis10.1016/j.jneumeth.2003.10.009 ↗
42. Borys, M., Tokovarov, M., Wawrzyk, M., Wesołowska, K., Plechawska-Wójcik, M., Dmytruk, R., and Kaczorowska, M. (2017, January 23–25). An analysis of eye-tracking and electroencephalography data for cognitive load measurement during arithmetic tasks. Proceedings of the 2017 10th International Symposium on Advanced Topics in Electrical Engineering (ATEE), Bucharest, Romania.10.1109/ATEE.2017.7905130 ↗
43. EEG dataset of individuals with intellectual and developmental disorder and healthy controls under rest and music stimuli10.1016/j.dib.2020.105488 ↗
44. Eliminating Electroencephalogram Artefacts Using Independent Component Analysis10.18100/ijamec.99374 ↗
45. Evaluation and prediction mental workload in user interface of maritime operations using eye response10.1016/j.ergon.2019.03.002 ↗
46. Eye Tracking System with Blink Detection
47. The amplitude velocity ratio of blinks: A new method for monitoring drowsiness
48. Performance Evaluation of SVM and K-Nearest Neighbor Algorithm over Medical Data set
49. Random forests10.1023/A:1010933404324 ↗
50. Variable selection using random forests10.1016/j.patrec.2010.03.014 ↗
51. Sitting posture detection and recognition of aircraft passengers using machine learning10.1017/S0890060421000135 ↗
52. A methodology for part classification with supervised machine learning10.1017/S0890060418000197 ↗
53. Comparative analysis of machine learning algorithms for predicting standard time in a manufacturing environment
54. A Neural Network Approach for Assessing the Relationship between Grip Strength and Hand Anthropometry10.14311/NNW.2015.25.030 ↗
55. Soft computing applications in the field of human factors and ergonomics: A review of the past decade of research10.1016/j.apergo.2023.104132 ↗
56. Classification of Customer Reviews Using Machine Learning Algorithms10.1080/08839514.2021.1922843 ↗
57. Investigating the relationship between adverse events and infrastructure development in an active war theater using soft computing techniques10.1016/j.asoc.2014.09.028 ↗
58. Haykin, S. (1999). Neural Networks: A Comprehensive Foundation, Prentice Hall Inc.. [2nd ed.].
59. Performance analysis of modeling framework for prediction in wind farms employing artificial neural networks10.1007/s00500-013-1084-9 ↗
60. Zurada, J. (1992). Introduction to Artificial Neural Systems, West Publishing Company.
61. Fausett, L. (1994). Fundamentals of Neural Networks, Pearson.
62. A comprehensive survey on support vector machine classification: Applications, challenges and trends10.1016/j.neucom.2019.10.118 ↗
63. Support Vector Machines for classification and regression10.1039/B918972F ↗
64. Greedy function approximation: A gradient boosting machine10.1214/aos/1013203451 ↗
65. Kuhn, M., and Johnson, K. (2013). Applied Predictive Modeling, Springer.10.1007/978-1-4614-6849-3 ↗
66. Chen, T., and Guestrin, C. (2016, January 13–17). Xgboost: A scalable tree boosting system. Proceedings of the 22nd Acm Sigkdd International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939785 ↗
67. Predicting the percentage of student placement: A comparative study of machine learning algorithms10.1007/s10639-021-10655-4 ↗
68. LightGBM: A highly efficient gradient boosting decision tree