journal article·2025-07-03Open access

FF-YOLO: An Improved YOLO11-Based Fatigue Detection Algorithm for Air Traffic Controllers

Shijie Tan, Weijun Pan, Leilei Deng, Qinghai Zuo, Yao Zheng

Applied Sciences· Volume 15· Issue 13

Abstract

Real-time detection of fatigue states in air traffic controllers (ATCOs) is crucial for ensuring air traffic safety. Existing methods exhibit limitations such as poor real-time performance, intrusiveness, and susceptibility to lighting and occlusion. This paper proposes FF-YOLO, an improved YOLO11-based deep learning algorithm, to detect ATCO fatigue states through facial feature analysis. A custom dataset comprising 25,154 facial images collected from 10 ATCOs was constructed for model training and validation. The FF-YOLO model introduces the CA-C3K2 module for fine-grained feature extraction under complex lighting, incorporates a spatial–channel attention mechanism for improved detection accuracy during occlusion, and MPDIoU loss for enhanced accuracy on multi-scale facial images with accelerated convergence. Experimental results show FF-YOLO achieves 94.2% mAP@50, 74.7% mAP@50–95, 83.8% precision, and 73.8% recall, with gains of +13.7%, +11.6%, +0.6%, and +5.9% over YOLO11n, respectively, thereby enabling real-time and accurate detection of ATCO fatigue states. Future work will expand datasets with larger and more diverse ATCO populations to enhance generalizability.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Vehicle Dynamics and Control Systems · Fatigue and fracture mechanics

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

College of Air Traffic Management, Civil Aviation Flight University of China, Guanghan 618307, China

Flight Technology and Flight Safety Research Base of the Civil Aviation Administration of China, Civil Aviation Flight University of China, Guanghan 618307, China

School of Electronic and Information Engineering, Beihang University, Beijing 100191, China

Funding

National Natural Science Foundation of China · Key Laboratory of Flight Techniques and Flight Safety CAAC · Fundamental Research Funds for the Central Universities · Civil Aircraft Fire Science and Safety Engineering Key Laboratory of Sichuan Province

References (42)

Show source-provided references
1. Schuver-van Blanken, M., Huisman, H., and Roerdink, M. (2010, January 20). The ATC Cognitive Process & Operational Situation Model. Proceedings of the European Association for Aviation Psychology Conference, Budapest, Hungary.
2. Zhang, J., Chen, Z., Liu, W., Ding, P., and Wu, Q. (2021). A field study of work type influence on air traffic controllers’ fatigue based on data-driven PERCLOS detection. Int. J. Environ. Res. Public Health, 18.10.3390/ijerph182211937 ↗
3. The analysis of occurrences associated with air traffic volume and air traffic controllers’ alertness for fatigue risk management10.1111/risa.13594 ↗
4. Pan, H., Hu, Y., Wang, Y., and Duong, V. (2024). Fatigue Detection in Air Traffic Controllers: A Comprehensive Review, IEEE Access.10.1109/ACCESS.2024.3513292 ↗
5. Mélan, C., and Cascino, N. (2018, January 24–28). Contrasting effects of work schedule changes and air traffic intensity on ATCOs’ fatigue, stress and quality of life. Proceedings of the 33rd Conference of the European Association of Aviation Psychology, Dubrovnik, Croatia.
6. Identifying flight fatigue factors: An econometric modeling approach10.1016/j.jairtraman.2005.05.006 ↗
7. Pettersson, M., and Westgren, O. (2013). Staff Scheduling in ACC at ATCC Stockholm. [Master’s Thesis, Linköping University Electronic Press].
8. Development of a fatigue scale10.1016/0022-3999(93)90081-P ↗
9. Assessment of human fatigue during physical performance using physiological signals: A review10.13005/bpj/1308 ↗
10. Liang, Q., Xu, L., Bao, N., Qi, L., Shi, J., Yang, Y., and Yao, Y. (2019). Research on non-contact monitoring system for human physiological signal and body movement. Biosensors, 9.10.3390/bios9020058 ↗
11. A comprehensive review of approaches to detect fatigue using machine learning techniques
12. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research10.1016/S0166-4115(08)62386-9 ↗
13. Triyanti, V., Azis, H.A., and Iridiastadi, H. (2020, January 17–19). Workload and fatigue assessment on air traffic controller. Proceedings of the 12th ISIEM (International Seminar on Industrial Engineering & Management): “Industrial Intelligence System on Engineering, Information, and Management”, Batu, Malang, East Java, Indonesia. No. 1.10.1088/1757-899X/847/1/012087 ↗
14. EEG-Based driver Fatigue Detection using Spatio-Temporal Fusion network with brain region partitioning strategy10.1109/TITS.2023.3348517 ↗
15. Intelligent recognition of fatigue and sleepiness based on inceptionV3-LSTM via multi-feature fusion10.1109/ACCESS.2020.3014508 ↗
16. Mu, S., Liao, S., Tao, K., and Shen, Y. (2024). Intelligent fatigue detection based on hierarchical multi-scale ECG representations and HRV measures. Biomed. Signal Process. Control, 92.10.1016/j.bspc.2024.106127 ↗
17. Using EEG and eye-tracking as indicators to investigate situation awareness variation during flight monitoring in air traffic control system10.1017/S0373463325000037 ↗
18. Fu, S., Yang, Z., Ma, Y., Li, Z., Xu, L., and Zhou, H. (2024). Advancements in the intelligent detection of driver fatigue and distraction: A comprehensive review. Appl. Sci., 14.10.3390/app14073016 ↗
19. Research on fatigue detection based on visual features10.1049/ipr2.12207 ↗
20. Chen, Z., Zhang, X., Li, J., Ni, J., Chen, G., Wang, S., Fan, F., Wang, C.C., and Li, X. (2021). Machine vision detection to daily facial fatigue with a nonlocal 3D attention network. arXiv.
21. An effective framework for driver fatigue recognition based on intelligent facial expressions analysis10.1109/ACCESS.2018.2878601 ↗
22. Driver fatigue detection based on improved YOLOv710.1007/s11554-024-01455-3 ↗
23. Lightweight YOLOM-Net for Automatic Identification and Real-Time Detection of Fatigue Driving
24. Yu, N., Yin, X., Li, B., Zhang, X., and Meng, J. (2024, January 8–10). Fatigue Driving Recognition Method Based on Improved GD-YOLO. Proceedings of the 2024 8th Asian Conference on Artificial Intelligence Technology (ACAIT), Fuzhou, China.10.1109/ACAIT63902.2024.11022159 ↗
25. The relationship between workload, performance and fatigue in a short-haul airline10.1080/07420528.2020.1804924 ↗
26. How high is high? A meta-analysis of NASA-TLX global workload scores10.1177/1541931215591373 ↗
27. Wierwille, W., Tijerina, L., Glecker, M., Duane, S., Johnston, S., and Goodman, M. (1998). PERCLOS: A Valid Psychophysiological Measure of Alertness As Assessed by Psychomotor Vigilance, United States Federal Motor Carrier Safety Administration.
28. Dlib-ml: A machine learning toolkit
29. Soukupova, T., and Cech, J. (2016, January 3–5). Eye blink detection using facial landmarks. Proceedings of the 21st Computer Vision Winter Workshop, Rimske Toplice, Slovenia.
30. Evaluation of driver drowsiness by trained raters10.1016/0001-4575(94)90019-1 ↗
31. Classification of imbalanced data: Review of methods and applications10.1088/1757-899X/1099/1/012077 ↗
32. Hybrid facial image feature extraction and recognition for non-invasive chronic fatigue syndrome diagnosis10.1016/j.compbiomed.2015.06.005 ↗
33. Hu, J., Shen, L., and Sun, G. (2018, January 18–22). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
34. Khanam, R., and Hussain, M. (2024). Yolov11: An overview of the key architectural enhancements. arXiv.
35. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
36. Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., and Ren, D. (2020, January 7–12). Distance-IoU loss: Faster and better learning for bounding box regression. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA. No. 7.
37. Du, S., Zhang, B., Zhang, P., and Xiang, P. (2021, January 16–18). An improved bounding box regression loss function based on CIOU loss for multi-scale object detection. Proceedings of the 2021 IEEE 2nd International Conference on Pattern Recognition and Machine Learning (PRML), Chengdu, China.10.1109/PRML52754.2021.9520717 ↗
38. Ma, S., and Xu, Y. (2023). Mpdiou: A loss for efficient and accurate bounding box regression. arXiv.
39. A statistical approach to machine translation
40. Schütze, H., Manning, C.D., and Raghavan, P. (2008). Introduction to Information Retrieval, Cambridge University Press.10.1017/CBO9780511809071 ↗
41. Davis, J., and Goadrich, M. (2006, January 25–29). The relationship between Precision-Recall and ROC curves. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA.10.1145/1143844.1143874 ↗
42. Lin, T.Y., Maire, M., Belongie, S., Hays, J., and Zitnick, C.L. (2014). Microsoft coco: Common objects in context, European Conf. Computer Vision, Springer.10.1007/978-3-319-10602-1_48 ↗