Driver Intention Recognition for Mine Transport Vehicle Based on Cross-Modal Knowledge Distillation
Source abstract
Abstract
Driver intention recognition is essential for optimizing driving decisions by dynamically adjusting speed and trajectory to enhance system performance. However, in the underground coal mine environment, traditional vision-based methods face significant limitations in accuracy and adaptability. To effectively improve the accuracy of vision-based driver intention recognition, this study introduces a novel approach leveraging cross-modal knowledge distillation (CMKD) to integrate electroencephalography (EEG) signals with video data to identify driver intentions in coal mining operations. By combining these modalities, the method capitalizes on their complementary strengths to achieve a more comprehensive understanding of driver intent. Experimental analysis across various models evaluates the performance of the proposed CMKD method, which integrates EEG signals with video data. Results reveal a substantial improvement in recognition accuracy over traditional machine vision-based approaches, with a maximum accuracy of 84.38%. This advancement enhances the reliability of driver intention detection and offers more robust support for decision making in automated mine transport systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Advanced Neural Network Applications · Autonomous Vehicle Technology and Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Mechanical and Electrical Engineering, China University of Mining and Technology (Beijing), Beijing 100083, China
Funding
National Key R&D Program of China · National Natural Science Foundation of China · 111 Project
References (46)
Show source-provided references
1. Low-Carbon Routing Based on Improved Artificial Bee Colony Algorithm for Electric Trackless Rubber-Tyred Vehicles10.23919/CSMS.2023.0011 ↗
2. Trivedi, M., Gandhi, T., and McCall, J. (2005, January 14–16). Looking-in and looking-out of a vehicle: Selected investigations in computer vision based enhanced vehicle safety. Proceedings of the IEEE International Conference on Vehicular Electronics and Safety, Xi’an, China.
3. A visual reasoning-based approach for driving experience improvement in the AR-assisted head-up displays10.1016/j.aei.2023.101888 ↗
4. STRUCTURE-FROM-MOTION FOR CALIBRATION OF A VEHICLE CAMERA SYSTEM WITH NON-OVERLAPPING FIELDS-OF-VIEW IN AN URBAN ENVIRONMENT10.5194/isprs-archives-XLII-1-W1-181-2017 ↗
5. Geo-Context Aware Study of Vision-Based Autonomous Driving Models and Spatial Video Data10.1109/TVCG.2021.3114853 ↗
6. Xu, H., Gao, Y., Yu, F., and Darrell, T. (2017, January 21–26). End-to-End Learning of Driving Models from Large-Scale Video Datasets. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.376 ↗
7. Ma, H., Wang, Y., Xiong, R., Kodagoda, S., and Tang, L. (2019). DeepGoal: Learning to Drive with driving intention from Human Control Demonstration. arXiv.10.1016/j.robot.2020.103477 ↗
8. Frossard, D., Kee, E., and Urtasun, R. (2019, January 20–24). DeepSignals: Predicting Intent of Drivers Through Visual Signals. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.10.1109/ICRA.2019.8794214 ↗
9. DIPNet: Driver intention prediction for a safe takeover transition in autonomous vehicles10.1049/itr2.12370 ↗
10. The Research of Coal Mine Underground Rubber Tyred Vehicle Wireless Video Aided Scheduling System
11. Xue, G., Li, R., Liu, S., and Wei, J. (2022). Research on Underground Coal Mine Map Construction Method Based on LeGO-LOAM Improved Algorithm. Energies, 15.10.3390/en15176256 ↗
12. A new network model for multiple object detection forautonomous vehicle detection in mining environment10.1049/ipr2.13173 ↗
13. A low-light image enhancement method for personnel safety monitoring in underground coal mines10.1007/s40747-024-01387-2 ↗
14. Affanni, A., and Najafi, T. (2022). Drivers’ Attention Assessment by Blink Rate Measurement from EEG Signals. 2022 IEEE International Workshop on Metrology for Automotive (MetroAutomotive), IEEE.10.1109/MetroAutomotive54295.2022.9855098 ↗
15. Zhang, X., Xiao, X., Yang, Y., Hao, Z., Li, J., and Huang, H. (2023, January 24–28). EEG Signal Analysis for Early Detection of Critical Road Events and Emergency Response in Autonomous Driving. Proceedings of the 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), Bilbao, Spain.10.1109/ITSC57777.2023.10422160 ↗
16. Jang, M., and Oh, K. (2024). Development of an Integrated Longitudinal Control Algorithm for Autonomous Mobility with EEG-Based Driver Status Classification and Safety Index. Electronics, 13.10.3390/electronics13071374 ↗
17. Li, M., Wang, W., Liu, Z., Qiu, M., and Qu, D. (2022). Driver Behavior and Intention Recognition Based on Wavelet Denoising and Bayesian Theory. Sustainability, 14.10.3390/su14116901 ↗
18. Discrimination Between Normal Driving and Braking Intention from Driver’s Brain Signals10.1007/978-3-319-78723-7_11 ↗
19. Intent recognition of torso motion using wavelet transform feature extraction and linear discriminant analysis ensemble classification10.1016/j.bspc.2017.06.010 ↗
20. Research on Human-Computer Interaction Intention Recognition Based on EEG and Eye Movement10.1109/ACCESS.2020.3011740 ↗
21. EEGNet-MSD: A Sparse Convolutional Neural Network for Efficient EEG-Based Intent Decoding10.1109/JSEN.2023.3295407 ↗
22. A Novel Multiscale Dilated Convolution Neural Network with Gating Mechanism for Decoding Driving Intentions Based on EEG10.1109/TCDS.2023.3245042 ↗
23. Ju, J., Bi, L., and Feleke, A.G. (2022). Noninvasive neural signal-based detection of soft and emergency braking intentions of drivers. Biomed. Signal Process. Control, 72.10.1016/j.bspc.2021.103330 ↗
24. Liang, X., Yang, Y., Liu, Y., Liu, K., Liu, Y., and Zhou, Z. (2023). EEG-based emergency braking intention detection during simulated driving. BioMed. Eng. OnLine, 22.10.1186/s12938-023-01129-4 ↗
25. Driving EEG based multilayer dynamic brain network analysis for steering process10.1016/j.eswa.2022.118121 ↗
26. Distilling the Knowledge in a Neural Network
27. Chen, Z., Li, Z., Zhang, S., Fang, J., Jiang, Q., and Zhao, F. (2023, January 1–5). BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detection. Proceedings of the International Conference on Learning Representations (ICLR), Kigali, Rwanda.
28. Thoker, F.M., and Gall, J. (2019, January 22–25). Cross-Modal Knowledge Distillation for Action Recognition. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8802909 ↗
29. Li, Y., Wang, Y., and Cui, Z. (2023, January 17–24). Decoupled Multimodal Distilling for Emotion Recognition. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00641 ↗
30. Bano, S., Tonellotto, N., and Gotta, A. (2022, January 21–25). Drivers Stress Identification in Real-World Driving Tasks. Proceedings of the 2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Pisa, Italy.10.1109/PerComWorkshops53856.2022.9767455 ↗
31. Visual-to-EEG cross-modal knowledge distillation for continuous emotion recognition10.1016/j.patcog.2022.108833 ↗
32. Liang, J. (2012). EEG Analysis and BCI Research Based on Motor Imagery Under Driving Behavior. [Ph.D. Thesis, Hebei University of Technology].
33. Lopez-Paz, D., Bottou, L., Bernhard, S., and Vapnik, V. (2015). Unifying distillation and privileged information. arXiv.
34. Cavazza, J., Ahmed, W., Volpi, R., Morerio, P., Bossi, F., Willemse, C., Wykowska, A., and Murino, V. (2022). Understanding action concepts from videos and brain activity through subjects’ consensus. Sci. Rep., 12.10.1038/s41598-022-23067-2 ↗
35. Paszke, A., Gross, S., Massa, F., Lerer, A., and Chintala, S. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv.
36. Loshchilov, I., and Hutter, F. (2017). Decoupled weight decay regulation. arXiv.
37. Hara, K., Kataoka, H., and Satoh, Y. (2017, January 22–29). Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition. Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), Venice, Italy.10.1109/ICCVW.2017.373 ↗
38. Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., and Paluri, M. (2018, January 18–22). A Closer Look at Spatiotemporal Convolutions for Action Recognition. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00675 ↗
39. Donahue, J., Hendricks, L.A., Guadarrama, S., and Rohrbach, M. (2015, January 7–12). Long-term recurrent convolutional networks for visual recognition and description. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298878 ↗
40. Feichtenhofer, C., Fan, H., Malik, J., and He, K. (November, January 27). SlowFast Networks for Video Recognition. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00630 ↗
41. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
42. Bkassiny, M. (2022, January 13). A Deep Learning-based Signal Classification Approach for Spectrum Sensing using Long Short-Term Memory (LSTM) Networks. Proceedings of the 2022 6th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE), Yogyakarta, Indonesia.10.1109/ICITISEE57756.2022.10057728 ↗
43. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. arXiv.
44. Bai, S., Kolter, J.Z., and Koltun, V. (2018). An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv.
45. Kingma, D., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv.
46. Thakkar, B. (2025). Continuous variable analyses: Student’s t-test, Mann–Whitney U test, Wilcoxon signed-rank test. Translational Cardiology, Academic Press.10.1016/B978-0-323-91790-2.00052-6 ↗