journal article·2025-01-14Open access

Human Trajectory Imputation Model: A Hybrid Deep Learning Approach for Pedestrian Trajectory Imputation

Deb Kanti Barua, Mithun Halder, Shayanta Shopnil, Md. Motaharul Islam

Applied Sciences· Volume 15· Issue 2

Abstract

Pedestrian trajectories are crucial for self-driving cars to plan their paths effectively. The sensors implanted in these self-driving vehicles, despite being state-of-the-art ones, often face inaccuracies in the perception of surrounding environments due to technical challenges in adverse weather conditions, interference from other vehicles’ sensors and electronic devices, and signal reception failure, leading to incompleteness in the trajectory data. But for real-time decision making for autonomous driving, trajectory imputation is no less crucial. Previous attempts to address this issue, such as statistical inference and machine learning approaches, have shown promise. Yet, the landscape of deep learning is rapidly evolving, with new and more robust models emerging. In this research, we have proposed an encoder–decoder architecture, the Human Trajectory Imputation Model, coined HTIM, to tackle these challenges. This architecture aims to fill in the missing parts of pedestrian trajectories. The model is evaluated using the Intersection drone the inD dataset, containing trajectory data at suitable altitudes, preserving naturalistic pedestrian behavior with varied dataset sizes. To assess the effectiveness of our model, we utilize L1, MSE, and quantile and ADE loss. Our experiments demonstrate that HTIM outperforms the majority of the state-of-the-art methods in this field, thus indicating its superior performance in imputing pedestrian trajectories.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Autonomous Vehicle Technology and Safety · Video Surveillance and Tracking Methods · Traffic and Road Safety

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Computer Science and Engineering, United International University, United City, Madani Avenue, Dhaka 1212, Bangladesh

Funding

United International University (UIU) Institute of Advanced Research (IAR)

References (34)

Show source-provided references
1. A New Approach to Linear Filtering and Prediction Problems10.1115/1.3662552 ↗
2. Dellaert, F., Fox, D., Burgard, W., and Thrun, S. (1999, January 10–15). Monte Carlo localization for mobile robots. Proceedings of the 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C), Detroit, MI, USA.
3. Kao, M.Y. (2008). Support Vector Machines. Encyclopedia of Algorithms, Springer.10.1007/978-0-387-30162-4 ↗
4. Goodfellow, I.J., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Available online: http://www.deeplearningbook.org.
5. Duch, W., Kacprzyk, J., Oja, E., and Zadrożny, S. (2005, January 11–15). Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition. Proceedings of the Artificial Neural Networks: Formal Models and Their Applications–ICANN 2005, Warsaw, Poland.10.1007/11550907 ↗
6. Attention-based LSTM (AttLSTM) neural network for Seismic Response Modeling of Bridges10.1016/j.compstruc.2022.106915 ↗
7. Ullah, M., Ullah, H., Khan, S.D., and Cheikh, F.A. (2019, January 28–31). Stacked Lstm Network for Human Activity Recognition Using Smartphone Data. Proceedings of the 2019 8th European Workshop on Visual Information Processing (EUVIP), Rome, Italy.10.1109/EUVIP47703.2019.8946180 ↗
8. Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.-k., and chun Woo, W. (2015). Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. arXiv.
9. Hua, G., and Jégou, H. (15–16, January 8–10). Temporal Convolutional Networks: A Unified Approach to Action Segmentation. Proceedings of the Computer Vision–ECCV 2016 Workshops, Amsterdam, The Netherlands.
10. Deng, B., Yan, J., and Lin, D. (2017). Peephole: Predicting Network Performance Before Training. arXiv.
11. Cao, W., Wang, D., Li, J., Zhou, H., Li, L., and Li, Y. (2018). BRITS: Bidirectional recurrent imputation for time series. Advances in Neural Information Processing Systems 31, Curran Associates, Inc.
12. The effects of the irregular sample and missing data in time series analysis
13. Nawaz, A., Huang, Z., Wang, S., Akbar, A., AlSalman, H., and Gumaei, A. (2020). GPS trajectory completion using end-to-end bidirectional convolutional recurrent encoder-decoder architecture with attention mechanism. Sensors, 20.10.3390/s20185143 ↗
14. Zheng, Y., Liu, L., Wang, L., and Xie, X. (2008, January 21–25). Learning Transportation Modes from Raw GPS Data for Geographic Application on the Web. Proceedings of the International Conference on World Wide Web (WWW 2008), Beijing, China.10.1145/1367497.1367532 ↗
15. Zheng, Y., Li, Q., Chen, Y., and Xie, X. (2008, January 21–24). Understanding Mobility Based on GPS Data. Proceedings of the ACM Conference on Ubiquitous Computing (UbiComp 2008), Seoul, Republic of Korea.10.1145/1409635.1409677 ↗
16. Understanding Transportation Modes Based on GPS Data for Web Applications10.1145/1658373.1658374 ↗
17. Ma, J., Yang, C., Mao, S., Zhang, J., Periaswamy, S.C., and Patton, J. (2022, January 16–20). Human Trajectory Completion with Transformers. Proceedings of the ICC 2022-IEEE International Conference on Communications, Seoul, Republic of Korea.10.1109/ICC45855.2022.9838743 ↗
18. Sikeridis, D., Papapanagiotou, I., and Devetsikiotis, M. (2024, January 10). CRAWDAD Unm/Blebeacon. Available online: https://ieee-dataport.org/open-access/crawdad-unmblebeacon.
19. A two-block rnn-based trajectory prediction from incomplete trajectory10.1109/ACCESS.2021.3072135 ↗
20. Pellegrini, S., Ess, A., Schindler, K., and van Gool, L. (October, January 27). You’ll never walk alone: Modeling social behavior for multi-target tracking. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.10.1109/ICCV.2009.5459260 ↗
21. Crowds by example
22. Move, Attend and Predict: An attention-based neural model for people’s movement prediction10.1016/j.patrec.2018.05.015 ↗
23. Yang, D., Zhang, D., Yu, Z., and Yu, Z. (2013, January 8–12). Fine-grained preference-aware location search leveraging crowdsourced digital footprints from LBSNs. Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Zurich, Switzerland.10.1145/2493432.2493464 ↗
24. Wang, Z., Zhang, S., and Yu, J.J. (2020, January 20–23). Reconstruction of Missing Trajectory Data: A Deep Learning Approach. Proceedings of the 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), Virtual.10.1109/ITSC45102.2020.9294402 ↗
25. Attention mechanism in neural networks: Where it comes and where it goes10.1007/s00521-022-07366-3 ↗
26. Enhanced Spatial–Temporal Savitzky–Golay Method for Reconstructing High-Quality NDVI Time Series: Reduced Sensitivity to Quality Flags and Improved Computational Efficiency
27. Leibe, B., Matas, J., Sebe, N., and Welling, M. (15–16, January 8–10). Learning Social Etiquette: Human Trajectory Understanding in Crowded Scenes. Proceedings of the Computer Vision–ECCV 2016, Amsterdam, The Netherlands.
28. Yang, D., Li, L., Redmill, K., and Ozguner, U. (2019, January 9–12). Top-view Trajectories: A Pedestrian Dataset of Vehicle-Crowd Interaction from Controlled Experiments and Crowded Campus. Proceedings of the 2019 IEEE Intelligent Vehicles Symposium (IV), Paris, France.10.1109/IVS.2019.8814092 ↗
29. Krajewski, R., Bock, J., Kloeker, L., and Eckstein, L. (2018, January 4–7). The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems. Proceedings of the 2018 21st International Conference on Intelligent Transportation Systems (ITSC), Maui, HI, USA.10.1109/ITSC.2018.8569552 ↗
30. Zhan, W., Sun, L., Wang, D., Shi, H., Clausse, A., Naumann, M., Kümmerle, J., Königshof, H., Stiller, C., and de La Fortelle, A. (2019). INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps. arXiv.
31. Strigel, E., Meissner, D., Seeliger, F., Wilking, B., and Dietmayer, K. (2014, January 8–11). The Ko-PER intersection laserscanner and video dataset. Proceedings of the 2014 17th IEEE International Conference on Intelligent Transportation Systems, ITSC 2014, Qingdao, China.10.1109/ITSC.2014.6957976 ↗
32. Bock, J., Krajewski, R., Moers, T., Runde, S., Vater, L., and Eckstein, L. (November, January 19). The inD Dataset: A Drone Dataset of Naturalistic Road User Trajectories at German Intersections. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA.10.1109/IV47402.2020.9304839 ↗
33. Muktadir, G.M., Ikram, Z., and Whitehead, J. (2023). Pedestrian Crossing Dataset Extraction from InD Dataset, preprint.
34. (2024, March 07). Available online: https://github.com/adhocmaster/drone-dataset-tools.