Multi-Scale Spatial–Temporal Transformer: A Novel Framework for Spatial–Temporal Edge Data Prediction
Source abstract
Abstract
Spatial–temporal prediction is an important part of a great number of applications, such as urban traffic control, urban traffic management, and urban traffic planning. However, real-world spatial–temporal data often have complex patterns, so it is still challenging to predict them accurately. Most existing spatial–temporal prediction models fail to aggregate the spatial features in a suitable neighborhood during fixed spatial dependencies extraction and lack adequately comprehensive time series analysis for intricate temporal dependencies. This paper proposes a novel model named multi-scale spatial–temporal transformer network (MSSTTN) to deal with intricate spatial–temporal patterns. Firstly, we develop an improved graph wavelet neural network, which learns how to pass the spatial graph signals of different frequency scales to adjust the neighborhood of feature aggregation adaptively. Then, we propose decomposing the time series into local trend-cyclical parts of various scales during time series analysis, making the model capture more reliable temporal dependencies. The proposed model has been evaluated on publicly available real-world datasets. The experimental findings indicate that the proposed model exhibits superior performance compared to conventional techniques including, spatial–temporal transformer (STTNs), GraphWaveNet, and others.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Traffic Prediction and Management Techniques · Human Mobility and Location-Based Analysis · Transportation Planning and Optimization
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science, China University of Geosciences, Wuhan 430074, China
Funding
NSFC · Hubei Provincial Department of Natural Resources
References (40)
Show source-provided references
1. A review of travel time estimation and forecasting for Advanced Traveller Information Systems
2. A summary of traffic flow forecasting methods
3. Dynamic modeling of urban transportation networks and analysis of its travel behaviors
4. Synthesis of short-term traffic flow forecasting research progress
5. A survey of machine learning and deep learning in remote sensing of geological environment: Challenges, advances, and opportunities10.1016/j.isprsjprs.2023.05.032 ↗
6. Integrating granger causality and vector autoregression for traffic prediction of large-scale wlans
7. Short-term traffic flow forecasting: An experimental comparison of time-series analysis and supervised learning10.1109/TITS.2013.2247040 ↗
8. Short-term traffic and travel time prediction models
9. Supervised weighting-online learning algorithm for short-term traffic flow prediction10.1109/TITS.2013.2267735 ↗
10. Zarei, N., Ghayour, M.A., and Hashemi, S. (2013, January 18–20). Road traffic prediction using context-aware random forest based on volatility nature of traffic flows. Proceedings of the Asian Conference on Intelligent Information & Database Systems, Kuala Lumpur, Malaysia.10.1007/978-3-642-36546-1_21 ↗
11. Freeway performance measurement: Mining loop detector data
12. Auto-regressive model for nonstationary stochastic processes10.1061/(ASCE)0733-9399(1988)114:11(1995) ↗
13. Jordan, M.I. (1986). Serial Order: A Parallel Distributed Processing Approach. Inst. Cogn. Sci. Rep.
14. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
15. Cho, K., Merrienboer, B.V., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv.10.3115/v1/D14-1179 ↗
16. Li, Y., Yu, R., Shahabi, C., and Liu, Y. (2017). Diffusion convolutional recurrent neural network: Data driven traffic forecasting. arXiv.
17. Tian, C., Ma, J., Zhang, C., and Zhan, P. (2018). A deep neural network model for short-term load forecast based on long short-term memory network and convolutional neural network. Energies, 11.10.3390/en11123493 ↗
18. Multiple convolutional neural networks for multivariate time series prediction10.1016/j.neucom.2019.05.023 ↗
19. Xu, M., Dai, W., Liu, C., Gao, X., Lin, W., Qi, G.J., and Xiong, H. (2020). Spatial-temporal transformer networks for traffic flow forecasting. arXiv.
20. Learning dynamics and heterogeneity of spatialtemporal graph data for traffic forecasting10.1109/TKDE.2021.3056502 ↗
21. Li, S., Jin, X., Xuan, Y., Zhou, X., Chen, W., Wang, Y.X., and Yan, X. (2019). Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. arXiv.
22. Zhou, H., Zhang, S., Peng, J., Zhang, S., and Zhang, W. (2021, January 2–9). Informer: Beyond efficient transformer for long sequence time-series forecasting. Proceedings of the Association for the Advancement of Artificial Intelligence, Virtually.
23. Autoformer: Decomposition transformers with autocorrelation for long-term series forecasting
24. Yao, H., Fei, W., Ke, J., Tang, X., and Ye, J. (2018, January 2–7). Deep multi-view spatial-temporal network for taxi demand prediction. Proceedings of the Association for the Advancement of Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11836 ↗
25. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
26. Gori, M., Monfardini, G., and Scarselli, F. (August, January 31). A new model for learning in graph domains. Proceedings of the IEEE International Joint Conference on Neural Networks, Montreal, QC, Canada.
27. Defferrard, M., Bresson, X., and Vandergheynst, P. (2016). Convolutional neural networks on graphs with fast localized spectral filtering. arXiv.
28. T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Prediction10.1109/TITS.2019.2935152 ↗
29. Kipf, T.N., and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv.
30. Yu, B., Yin, H., and Zhu, Z. (2017). Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv.10.24963/ijcai.2018/505 ↗
31. Wu, Z., Pan, S., Long, G., Jiang, J., and Zhang, C. (2019). Graph wavenet for deep spatial-temporal graph modeling. arXiv.10.24963/ijcai.2019/264 ↗
32. Guo, S., Lin, Y., Feng, N., Song, C., and Wan, H. (February, January 27). Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. Proceedings of the National Conference on Artificial Intelligence, Honolulu, HI, USA.
33. Meta graph transformer: A novel framework for spatial–temporal traffic prediction10.1016/j.neucom.2021.12.033 ↗
34. Bruna, J., Zaremba, W., Szlam, A., and Lecun, Y. (2013). Spectral networks and locally connected networks on graphs. arXiv.
35. Xu, B., Shen, H., Cao, Q., Qiu, Y., and Cheng, X. (2019). Graph wavelet neural network. arXiv.
36. Donnat, C., Zitnik, M., Hallac, D., and Leskovec, J. (2018). Learning structural node embeddings via diffusion wavelets. arXiv.10.1145/3219819.3220025 ↗
37. Wavelets on graphs via spectral graph theory10.1016/j.acha.2010.04.005 ↗
38. Sutskever, I., Vinyals, O., and Le, Q.V. (2014). Sequence to sequence learning with neural networks. arXiv.
39. Rao, X., Wang, H., Zhang, L., Li, J., Shang, S., and Han, P. (2022, January 23–29). FOGS: First-Order Gradient Supervision with Learning-based Graph for Traffc Flow Forecasting. Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, Vienna, Austria.10.24963/ijcai.2022/545 ↗
40. Kingma, D., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv.