Workload Assessment of Operators: Correlation Between NASA-TLX and Pupillary Responses
Source abstract
Abstract
Operators in high-stress environments often face significant cognitive demands that can impair their performance, underscoring the need for comprehensive workload assessment. This study aims to study the relationship between subjective self-reported measures, the NASA task load index (NASA-TLX), objective bio-signal measures, and pupillary responses. The participants engaged in either a visual tracking task or a laparoscopic visuomotor task while their eye movements were recorded using a Tobii Pro Nano eye tracker (Tobii Technology Inc., Stockholm, Sweden). Immediately after completing the tasks, participants provided NASA-TLX scores to assess their perceived workload. The study tested three hypotheses: first, whether increased pupil dilation correlates with higher NASA-TLX scores; second, whether task type affects workload; and third, whether task repetition influences workload. The results showed a moderate positive correlation between pupil size and NASA-TLX scores (r = 0.513, p < 0.001). The laparoscopic surgery task, which requires visuomotor coordination, resulted in significantly higher NASA-TLX scores (t = –6.23, p < 0.001), larger original pupil sizes (t = –22.57, p < 0.001), and more adjusted pupil sizes (t = –22.57, p < 0.001) than the purely visual task. Additionally, task repetition led to a significant reduction in the NASA-TLX scores (t = 2.86, p = 0.005), the original mean pupil size (t = 5.50, p < 0.001), and the adjusted pupil size (t = 6.34, p < 0.001). In conclusion, the study confirms a positive correlation between NASA-TLX scores and pupillary responses. Task type and repetition were found to influence workload and pupillary responses. The findings demonstrate the value of using both subjective and objective measures for workload assessments.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human-Automation Interaction and Safety · Musculoskeletal pain and rehabilitation · Healthcare Technology and Patient Monitoring
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Surgery, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, AB T6G 2S2, Canada
Funding
China Scholarship Council
References (52)
Show source-provided references
1. A review of cognitive support systems in the operating room10.1177/15533506231218962 ↗
2. Effects of cognitive load on driving performance: The cognitive control hypothesis10.1177/0018720817690639 ↗
3. Cognitive load theory, learning difficulty, and instructional design10.1016/0959-4752(94)90003-5 ↗
4. Cognitive load management: An invaluable tool for safe and effective surgical training10.1016/j.jsurg.2022.12.010 ↗
5. The impact of workload on job performance among doctors in Malaysian public hospitals: A case study
6. Impact of operator fatigue on endoscopy performance: Implications for procedure scheduling10.1007/s10620-008-0549-7 ↗
7. Impact of image-guided surgery on surgeons’ performance: A literature review10.1504/IJHFE.2016.083516 ↗
8. Hart, S.G. (2006, January 16–20). NASA-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, San Francisco, CA, USA.10.1037/e577632012-009 ↗
9. Kramer, A.F. (2020). Physiological Metrics of Mental Workload: A Review of Recent Progress. Multiple Task Performance, CRC Press.10.1201/9781003069447-14 ↗
10. Pupil dilation as an index of effort in cognitive control tasks: A review10.3758/s13423-018-1432-y ↗
11. Detection of changes in surgical difficulty: Evidence from pupil responses10.1177/1553350615573582 ↗
12. Jiang, X., Atkins, M.S., Tien, G., Bednarik, R., and Zheng, B. (May, January 26). Pupil Responses During Discrete Goal-Directed Movements. Proceedings of the SIGCHI Conference on Human Factors Computing Systems, Toronto, ON, Canada.
13. Multi-modal indicators for estimating perceived cognitive load in post-editing of machine translation10.1007/s10590-019-09227-8 ↗
14. Complexity in declarative process models: Metrics and multi-modal assessment of cognitive load10.1016/j.eswa.2023.120924 ↗
15. Bednarik, R., Bartczak, P., Vrzakova, H., Koskinen, J., Elomaa, A.-P., Huotarinen, A., de Gómez Pérez, D.G., and von und zu Fraunberg, M. (2018, January 14–17). Pupil Size as an Indicator of Visual-Motor Workload and Expertise in Microsurgical Training Tasks. Proceedings of the Symposium on Eye Tracking Research & Applications, Warsaw, Poland.10.1145/3204493.3204577 ↗
16. From theory to practice: The application of cognitive load theory to the practice of medicine10.1097/ACM.0000000000003524 ↗
17. Workload assessment of surgeons: Correlation between NASA TLX and blinks10.1007/s00464-012-2268-6 ↗
18. Solhjoo, S., Haigney, M.C., McBee, E., van Merrienboer, J.J.G., Schuwirth, L., Artino, A.R., Battista, A., Ratcliffe, T.A., Lee, H.D., and Durning, S.J. (2019). Heart rate and heart rate variability correlate with clinical reasoning performance and self-reported measures of cognitive load. Sci. Rep., 9.10.1038/s41598-019-50280-3 ↗
19. Respiratory changes in response to cognitive load: A systematic review10.1155/2016/8146809 ↗
20. Buchwald, M., Kupinski, S., Bykowski, A., Marcinkowska, J., Ratajczyk, D., and Jukiewicz, M. (2019, January 18–20). Electrodermal Activity as a Measure of Cognitive Load: A Methodological Approach. Proceedings of the 2019 Signal Processing: Algorithms, Architectures, Arrangements, and Applications, SPA, Poznan, Poland.10.23919/SPA.2019.8936745 ↗
21. Advancing the understanding of pupil size variation in occupational safety and health: A systematic review and evaluation of open-source methodologies10.1016/j.ssci.2024.106490 ↗
22. The measurement of cognitive workload in surgery using pupil metrics: A systematic review and narrative analysis10.1016/j.jss.2022.07.010 ↗
23. Investigation of eye movement measures of mental workload in healthcare: Can pupil dilations reflect fatigue or overload when it comes to health information system use?10.1016/j.apergo.2023.104150 ↗
24. Mohanty, S., Kim, J.H., Pulipati, V., Wang, F., Mostowfi, S., Oprean, D., Wang, Y., and Seo, K. (July, January 29). Measuring Cognitive Workload in Augmented Reality Learning Environments Through Pupil Area Analysis. Proceedings of the International Conference on Human-Computer Interaction, Washington, DC, USA.
25. Cognitive workload affects ocular accommodation and pupillary response10.1016/j.optom.2022.05.001 ↗
26. Gorin, H., Patel, J., Qiu, Q., Merians, A., Adamovich, S., and Fluet, G. (2024). A Review of the Use of Gaze and Pupil Metrics to Assess Mental Workload in Gamified and Simulated Sensorimotor Tasks. Sensors, 24.10.3390/s24061759 ↗
27. Studying Developer Eye Movements to Measure Cognitive Workload and Visual Effort for Expertise Assessment10.1145/3591135 ↗
28. Using eye tracking to examine expert-novice differences during simulated surgical training: A case study10.1016/j.chb.2023.107720 ↗
29. Schirm, J., Gómez-Vargas, A.R., Perusquía-Hernández, M., Skarbez, R.T., Isoyama, N., Uchiyama, H., and Kiyokawa, K. (2023). Identification of Language-Induced Mental Load from Eye Behaviors in Virtual Reality. Sensors, 23.10.3390/s23156667 ↗
30. Cognitive load in electromagnetic navigational and robotic bronchoscopy for pulmonary nodules10.34197/ats-scholar.2020-0033OC ↗
31. Pupillary responses for cognitive load measurement to classify difficulty levels in an educational video game: Empirical study10.2196/21620 ↗
32. Improving pilot mental workload evaluation with combined measures10.3233/BME-141041 ↗
33. Distracted worker: Using pupil size and blink rate to detect cognitive load during manufacturing tasks10.1016/j.apergo.2022.103867 ↗
34. Differences in gaze behaviour of expert and junior surgeons performing open inguinal hernia repair10.1007/s00464-014-3683-7 ↗
35. Combined gaze metrics as stress-sensitive indicators of microsurgical proficiency10.1177/1553350620942980 ↗
36. Eye-tracking metrics predict perceived workload in robotic surgical skills training10.1177/0018720819874544 ↗
37. Patel, A.N., Chau, G., Chang, C., Sun, A., Huang, J., Jung, T.P., and Gilja, V. (2021, January 1–5). Affective Response to Volitional Input Perturbations in Obstacle Avoidance and Target Tracking Games. Proceedings of the 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Virtual Conference.10.1109/EMBC46164.2021.9630523 ↗
38. Physiological parameter response to variation of mental workload10.1177/0018720817733101 ↗
39. Köles, M., Szegletes, L., and Forstner, B. (2015, January 19–21). Towards a Physiology-Based Difficulty Control System for Serious Games. Proceedings of the 2015 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), Győr, Hungary.10.1109/CogInfoCom.2015.7390612 ↗
40. Quantitative evaluations of the effects of noise on mental workloads based on pupil dilation during laparoscopic surgery10.1177/000313481808401243 ↗
41. Subjective and objective quantification of the effect of distraction on physician’s workload and performance during simulated laparoscopic surgery10.12659/MSM.914635 ↗
42. Wu, Y., Zhang, Z., Aghazadeh, F., and Zheng, B. (2024). Early Eye Disengagement Is Regulated by Task Complexity and Task Repetition in Visual Tracking Task. Sensors, 24.10.3390/s24102984 ↗
43. BORIS: A free, versatile open-source event-logging software for video/audio coding and live observations10.1111/2041-210X.12584 ↗
44. The pupil as a measure of emotional arousal and autonomic activation10.1111/j.1469-8986.2008.00654.x ↗
45. How to assess sympathetic activity in humans10.1097/00004872-199917060-00001 ↗
46. Eye movements in surgery: A literature review10.16910/jemr.6.4.4 ↗
47. Schmidt, R., and Lee, T. (2019). Motor Learning and Performance, 6th Edition with Web Study Guide–Loose-Leaf Edition: From Principles to Application, Human Kinetics Publishers.
48. Godde, B., and Voelcker-Rehage, C. (2017). Cognitive resources necessary for motor control in older adults are reduced by walking and coordination training. Front. Hum. Neurosci., 11.10.3389/fnhum.2017.00156 ↗
49. The multiple effects of practice: Skill, habit and reduced cognitive load10.1016/j.cobeha.2018.01.015 ↗
50. Quantifying mental workloads of surgeons performing natural orifice transluminal endoscopic surgery (NOTES) procedures10.1007/s00464-011-2038-x ↗
51. Measuring mental workload during the performance of advanced laparoscopic tasks10.1007/s00464-009-0522-3 ↗
52. Duchowski, A.T., Krejtz, K., Gehrer, N.A., Bafna, T., and Bækgaard, P. (2020, January 25–30). The Low/High Index of Pupillary Activity. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.10.1145/3313831.3376394 ↗