A Financial Time-Series Prediction Model Based on Multiplex Attention and Linear Transformer Structure
Source abstract
Abstract
Financial time-series prediction has been an important topic in deep learning, and the prediction of financial time series is of great importance to investors, commercial banks and regulators. This paper proposes a model based on multiplexed attention mechanisms and linear transformers to predict financial time series. The linear transformer model has a faster model training efficiency and a long-time forecasting capability. Using a linear transformer reduces the original transformer’s complexity and preserves the decoder’s multiplexed attention mechanism. The results show that the proposed method can effectively improve the prediction accuracy of the model, increase the inference speed of the model and reduce the number of operations, which has new implications for the prediction of financial time series.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Stock Market Forecasting Methods · Data Stream Mining Techniques · Energy Load and Power Forecasting
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Management, Wuhan Institute of Technology, Wuhan 430205, China
Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China
School of Computing, University of Portsmouth, Portsmouth PO1 3HE, UK
Funding
National Key R&D Program of China
References (41)
Show source-provided references
1. The use of ARIMA models for reliability forecasting and analysis10.1016/S0360-8352(98)00066-7 ↗
2. Dynamic forecasts of qualitative variables: A Qual VAR model of US recessions10.1198/073500104000000613 ↗
3. Non–Gaussian GARCH option pricing models and their diffusion limits10.1016/j.ejor.2015.06.046 ↗
4. Forecasting stock indices with back propagation neural network10.1016/j.eswa.2011.04.222 ↗
5. Financial time series forecasting using support vector machines10.1016/S0925-2312(03)00372-2 ↗
6. Luong, C., and Dokuchaev, N. (2018). Forecasting of realised volatility with the random forests algorithm. J. Risk Financ. Manag., 11.10.3390/jrfm11040061 ↗
7. JWSAA: Joint weak saliency and attention aware for person re-identification10.1016/j.neucom.2020.05.106 ↗
8. A Novel Hyperspectral Image Classification Model Using Bole Convolution with Three-Directions Attention Mechanism: Small sample and Unbalanced Learning
9. Multivariate cnn-lstm model for multiple parallel financial time-series prediction10.1155/2021/9903518 ↗
10. Probabilistic transformer for time series analysis
11. Hyper-sausage coverage function neuron model and learning algorithm for image classification10.1016/j.patcog.2022.109216 ↗
12. He, F., and Ye, Q. (2022). A bearing fault diagnosis method based on wavelet packet transform and convolutional neural network optimized by simulated annealing algorithm. Sensors, 22.10.20944/preprints202201.0068.v1 ↗
13. Chen, Z., Silvestri, F., Wang, J., Zhu, H., Ahn, H., and Tolomei, G. (2022, January 17–21). ReLAX: Reinforcement Learning Agent Explainer for Arbitrary Predictive Models. Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA.10.1145/3511808.3557429 ↗
14. Wei, X., and Saha, D. (2022, January 19). KNEW: Key Generation using NEural Networks from Wireless Channels. Proceedings of the 2022 ACM Workshop on Wireless Security and Machine Learning, San Antonio, TX, USA.10.1145/3522783.3529526 ↗
15. Heidari, M., and Rafatirad, S. (2020, January 12–15). Bidirectional transformer based on online text-based information to implement convolutional neural network model for secure business investment. Proceedings of the 2020 IEEE International Symposium on Technology and Society (ISTAS), Tempe, AZ, USA.10.1109/ISTAS50296.2020.9462170 ↗
16. Zou, Z.B., Song, L.P., and Song, Z.L. (2017, January 13–16). Labeled box-particle PHD filter for multi-target tracking. Proceedings of the 2017 3rd IEEE International Conference on Computer and Communications (ICCC), Chengdu, China.10.1109/CompComm.2017.8322835 ↗
17. Joint spatio-temporal precoding for practical non-stationary wireless channels10.1109/TCOMM.2023.3241326 ↗
18. Mohanrajan, S.N., and Loganathan, A. (2022). Novel vision transformer–based bi-LSTM model for LU/LC prediction—Javadi Hills, India. Appl. Sci., 12.10.3390/app12136387 ↗
19. Peng, H., Huang, S., Chen, S., Li, B., Geng, T., Li, A., Jiang, W., Wen, W., Bi, J., and Liu, H. (2022, January 10–14). A length adaptive algorithm-hardware co-design of transformer on fpga through sparse attention and dynamic pipelining. Proceedings of the 59th ACM/IEEE Design Automation Conference, San Francisco, CA, USA.10.1145/3489517.3530585 ↗
20. Fault diagnosis strategy of CNC machine tools based on cascading failure10.1007/s10845-017-1382-7 ↗
21. Determination of the average maintenance time of CNC machine tools based on type II failure correlation10.17531/ein.2017.4.15 ↗
22. Shen, G., Han, C., Chen, B., Dong, L., and Cao, P. (2018, January 22–24). Fault analysis of machine tools based on grey relational analysis and main factor analysis. Proceedings of the 3rd Annual International Conference on Information System and Artificial Intelligence (ISAI2018), Suzhou, China.
23. Shen, G.-x., Zhao, X.-z., Zhang, Y.-z., and Han, C.-y. (2017, January 23–24). Research on criticality analysis method of CNC machine tools components under fault rate correlation. Proceedings of the International Conference on Mechanical Engineering and Applied Composite Materials, Hong Kong, China.
24. Equitable healthcare access during the pandemic: The impact of digital divide and other sociodemographic and systemic factors
25. Relationship between racial bias exposure, financial literacy, and entrepreneurial intention: An empirical investigation
26. The Optimization of Supply Chain Financing for Bank Green Credit Using Stackelberg Game Theory in Digital Economy Under Internet of Things10.4018/JOEUC.333472 ↗
27. Hollis, T., Viscardi, A., and Yi, S.E. (2018). A comparison of LSTMs and attention mechanisms for forecasting financial time series. arXiv.
28. Research on the Relationship between CEO’s Overconfidence and Corporate Investment Financing Behavior10.32732/jmo.2018.10.1.8 ↗
29. The International Monetary Fund: A review of the recent evidence10.1007/s11558-007-9026-x ↗
30. The impact of COVID-19 on the standard & poor 500 index sectors: A multivariate generalized autoregressive conditional heteroscedasticity model10.1108/JCEFTS-08-2020-0049 ↗
31. Niu, H., Lin, Z., Zhang, X., and Jia, T. (2022, January 20–22). Image Segmentation For pneumothorax disease Based On based on Nested Unet Model. Proceedings of the 2022 3rd International Conference on Computer Vision, Image and Deep Learning & International Conference on Computer Engineering and Applications (CVIDL & ICCEA), Changchun, China.10.1109/CVIDLICCEA56201.2022.9824606 ↗
32. Li, Y., Wei, D., Chen, X., Song, Z., Wu, R., Li, Y., Jin, X., and Xu, W. (2018, January 23–26). Dumbnet: A smart data center network fabric with dumb switches. Proceedings of the Thirteenth EuroSys Conference, Porto, Portugal.10.1145/3190508.3190531 ↗
33. Li, C., Chen, Z., and Jiao, Y. (2023). Vibration and Bandgap Behavior of Sandwich Pyramid Lattice Core Plate with Resonant Rings. Materials, 16.10.3390/ma16072730 ↗
34. Chen, B.R., Liu, Z., Song, J., Zeng, F., Zhu, Z., Bachu, S.P.K., and Hu, Y.C. (2022, January 6–9). FlowTele: Remotely shaping traffic on internet-scale networks. Proceedings of the 18th International Conference on emerging Networking EXperiments and Technologies, Roma, Italy.10.1145/3555050.3569139 ↗
35. Siami-Namini, S., Tavakoli, N., and Namin, A.S. (2019). A comparative analysis of forecasting financial time series using arima, lstm, and bilstm. arXiv.
36. Adversarial sparse transformer for time series forecasting
37. Using support vector machines in financial time series forecasting
38. Siami-Namini, S., and Namin, A.S. (2018). Forecasting economics and financial time series: ARIMA vs. LSTM. arXiv.
39. Analysis of artificial neural network for financial time series forecasting
40. Financial time series forecasting model based on CEEMDAN and LSTM10.1016/j.physa.2018.11.061 ↗
41. The graph neural network model10.1109/TNN.2008.2005605 ↗