Forecasting Future Trajectories with an Improved Transformer Network
Source abstract
Abstract
An increase in car ownership brings convenience to people’s life. However, it also leads to frequent traffic accidents. Precisely forecasting surrounding agents’ future trajectories could effectively decrease vehicle-vehicle and vehicle-pedestrian collisions. Long-short-term memory (LSTM) network is often used for vehicle trajectory prediction, but it has some shortages such as gradient explosion and low efficiency. A trajectory prediction method based on an improved Transformer network is proposed to forecast agents’ future trajectories in a complex traffic environment. It realizes the transformation from sequential step processing of LSTM to parallel processing of Transformer based on attention mechanism. To perform trajectory prediction more efficiently, a probabilistic sparse self-attention mechanism is introduced to reduce attention complexity by reducing the number of queried values in the attention mechanism. Activate or not (ACON) activation function is adopted to select whether to activate or not, hence improving model flexibility. The proposed method is evaluated on the publicly available benchmarks next-generation simulation (NGSIM) and ETH/UCY. The experimental results indicate that the proposed method can accurately and efficiently predict agents’ trajectories.
Source: openalex_inverted_index · Quality: baseline
Research context
OpenAlex topics: Traffic Prediction and Management Techniques · Autonomous Vehicle Technology and Safety · Human Mobility and Location-Based Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (48)
Show source-provided references
1. Pedestrian motion trajectory prediction in intelligent driving from far shot first-person perspective video10.1109/TITS.2021.3052908 ↗
2. RSOD: Real-time small object detection algorithm in UAV-based traffic monitoring
3. Deformation expression of soft tissue based on BP neural network10.32604/iasc.2022.016543 ↗
4. A robust 3-D medical watermarking based on wavelet transform for data protection10.32604/csse.2022.022305 ↗
5. Robust reversible audio watermarking scheme for telemedicine and privacy protection10.32604/cmc.2022.022304 ↗
6. Seamless GPS/inertial navigation system based on self-learning square-root cubature Kalman filter10.1109/TIE.2020.2967671 ↗
7. Multi-rate strong tracking square-root cubature Kalman filter for MEMS-INS/GPS/polarization compass integrated navigation system10.1016/j.ymssp.2021.108146 ↗
8. A novel graph-based trajectory predictor with pseudo-oracle
9. IRLSOT: Inverse reinforcement learning for scene-oriented trajectory prediction10.1049/itr2.12172 ↗
10. An adversarial learned trajectory predictor with knowledge-rich latent variables
11. Vehicle trajectory prediction based on motion model and maneuver recognition
12. Intention recognition and trajectory prediction for vehicles using LSTM network
13. A lightweight CNN based on transfer learning for COVID-19 diagnosis10.32604/cmc.2022.024589 ↗
14. Anomalous behaviors detection in moving crowds based on a weighted convolutional autoencoder-long short-term memory network10.1109/TCDS.2018.2866838 ↗
15. LSTM network: A deep learning approach for short-term traffic forecast10.1049/iet-its.2016.0208 ↗
16. Surround vehicles trajectory analysis with recurrent neural networks
17. Generalizable intention prediction of human drivers at intersections
18. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network
19. Social LSTM: Human trajectory prediction in crowded spaces
20. J. Devlin, M. W. Chang, K. Lee and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” 2018. [Online]. Available: https://arxiv.org/abs/1810.04805.
21. Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma et al., “Albert: A lite bert for self-supervised learning of language representations,” 2019. [Online]. Available: https://arxiv.org/abs/1909.11942.
22. R. Child, S. Gray, A. Radford and I. Sutskever, “Generating long sequences with sparse transformers,” 2019. [Online]. Available: https://arxiv.org/abs/1904.10509.
23. A multi-feature learning model with enhanced local attention for vehicle re-identification10.32604/cmc.2021.021627 ↗
24. Spatio-temporal graph transformer networks for pedestrian trajectory prediction
25. Informer: Beyond efficient transformer for long sequence time-series forecasting
26. N. Kitaev, L. Kaiser and A. Levskaya, “Reformer: The efficient transformer,” 2020. [Online]. Available: https://arxiv.org/abs/2001.04451.
27. Activate or not: Learning customized activation
28. K. Ahmed, N. S. Keskar and R. Socher, “Weighted transformer network for machine translation,” 2017. [Online]. Available: https://arxiv.org/abs/1711.02132.
29. Transformer and multi-scale convolution fortarget-oriented sentiment analysis
30. Convolutional transformer with sentiment-aware attention for sentiment analysis
31. Enhanced transformer model for data-to-text generation
32. K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares et al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” 2014. [Online]. Available: https://arxiv.org/abs/1406.1078.10.3115/v1/D14-1179 ↗
33. Asynchronous bidirectional decoding for neural machine translation
34. Attention branch network: Learning of attention mechanism for visual explanation
35. TBE-Net: A three-branch embedding network with part-aware ability and feature complementary learning for vehicle re-identification10.1109/TITS.2021.3116045 ↗
36. DLnet with training task conversion stream for precise semantic segmentation in actual traffic scene
37. Multistability and instability analysis of recurrent neural networks with time-varying delays10.1016/j.neunet.2017.09.013 ↗
38. A comparison of deep networks with ReLU activation function and linear spline-type methods10.1016/j.neunet.2018.11.005 ↗
39. P. Ramachandran, B. Zoph and Q. V. Le, “Searching for activation functions,” 2017. [Online]. Available: https://arxiv.org/abs/1710.05941.
40. Unconstrained offline handwritten word recognition by position embedding integrated resnets model10.1109/LSP.2019.2895967 ↗
41. Loss function search for face recognition
42. The next generation simulation program
43. A critical evaluation of the Next Generation Simulation (NGSIM) vehicle trajectory dataset10.1016/j.trb.2017.09.018 ↗
44. Improving data association by joint modeling of pedestrian trajectories and groupings
45. Social GAN: Socially acceptable trajectories with generative adversarial networks
46. An attentive gan for predicting paths compliant to social and physical constraints
47. Crossing or not? Context-based recognition of pedestrian crossing intention in the urban environment10.1109/TITS.2021.3053031 ↗
48. Vehicle re-identification model based on optimized densenet121 with joint loss10.32604/cmc.2021.016560 ↗