A Stock Market Decision-Making Framework Based on CMR-DQN
Source abstract
Abstract
In the dynamic and uncertain stock market, precise forecasting and decision-making are crucial for profitability. Traditional deep neural networks (DNN) often struggle with capturing long-term dependencies and multi-scale features in complex financial time series data. To address these challenges, we introduce CMR-DQN, an innovative framework that integrates discrete wavelet transform (DWT) for multi-scale data analysis, temporal convolutional network (TCN) for extracting deep temporal features, and a GRU–LSTM–Attention mechanism to enhance the model’s focus and memory. Additionally, CMR-DQN employs the Rainbow DQN reinforcement learning strategy to learn optimal trading strategies in a simulated environment. CMR-DQN significantly improved the total return rate on six selected stocks, with increases ranging from 20.37% to 55.32%. It also demonstrated substantial improvements over the baseline model in terms of Sharpe ratio and maximum drawdown, indicating increased excess returns per unit of total risk and reduced investment risk. These results underscore the efficiency and effectiveness of CMR-DQN in handling multi-scale time series data and optimizing stock market decisions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Stock Market Forecasting Methods · Energy Load and Power Forecasting · Market Dynamics and Volatility
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information and Communication Engineering, Hainan University, Haikou 570228, China
School of Electronic Infromation, Central South University, Changsha 410083, China
Funding
Hainan University under Grant · National Key Research and Development Program of China · National Natural Science Foundation of China · Performance Computing Center of Central South University (HPC)
References (39)
Show source-provided references
1. Forecasting the changes in daily stock prices in Shanghai Stock Exchange using Neural Network and Ordinary Least Squares Regression
2. Dixon, F., Halperin, I., and Bilokon, P. (2020). Machine Learning in Finance: From Theory to Practice, Springer International Publishing.10.1007/978-3-030-41068-1 ↗
3. Recent advances in reinforcement learning in finance10.1111/mafi.12382 ↗
4. Reinforcement learning applied to Forex trading10.1016/j.asoc.2018.09.017 ↗
5. Koyano, S., and Ikeda, K. (December, January 27). Online portfolio selection based on the posts of winners and losers in stock microblogs. Proceedings of the 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, HI, USA.10.1109/SSCI.2017.8280902 ↗
6. Forward Forecast of Stock Price Using Sliding-Window Metaheuristic-Optimized Machine-Learning Regression10.1109/TII.2018.2794389 ↗
7. Tsai, M.C., Cheng, C.H., Tsai, M.T., and Shiu, H.Y. (2019). Forecasting leading industry stock prices based on a hybrid time-series forecast model. PloS ONE, 13.10.1371/journal.pone.0209922 ↗
8. Mugerman, Y., Winter, E., and Yafeh, T. (2023). Herding and Divergent Behaviors in Competition: An Experimental Study. SSRN Electron. J.10.2139/ssrn.4644677 ↗
9. Volodymyr, M., Koray, K., David, S., Alex, G., Ioannis, A., Daan, W., and Martin, A.R. (2013). Playing Atari with Deep Reinforcement Learning. arXiv.
10. Price Trailing for Financial Trading Using Deep Reinforcement Learning10.1109/TNNLS.2020.2997523 ↗
11. Mastering the game of Go with deep neural networks and tree search10.1038/nature16961 ↗
12. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
13. Deep Robust Reinforcement Learning for Practical Algorithmic Trading10.1109/ACCESS.2019.2932789 ↗
14. A parallel multi-module deep reinforcement learning algorithm for stock trading10.1016/j.neucom.2021.04.005 ↗
15. Multi-type data fusion framework based on deep reinforcement learning for algorithmic trading10.1007/s10489-022-03321-w ↗
16. Financial Stock Investment Management Using Deep Learning Algorithm in the Internet of Things
17. Huang, Y., Lu, X., Zhou, C., and Song, Y. (2023). DADE-DQN: Dual Action and Dual Environment Deep Q-Network for Enhancing Stock Trading Strategy. Mathematics, 11.10.3390/math11173626 ↗
18. Hasselt, H. (2010). Double Q-Learning, MIT Press.
19. Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized Experience Replay. arXiv.
20. Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., and Silver, D. (2018, January 2–7). Rainbow: Combining Improvements in Deep Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11796 ↗
21. Wang, Z., Schaul, T., Hessel, M., Hasselt, H.V., Lanctot, M., and Freitas, N. (2015, January 20–22). Dueling Network Architectures for Deep Reinforcement Learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA. Abstract Number 9.
22. Hausknecht, M., and Stone, P. (2015). Deep Recurrent Q-Learning for Partially Observable MDPs. arXiv.
23. Chen, X., Wang, Q., Yuxin, L., Hu, C., Wang, C., and Yan, Q. (2023, January 18–20). Stock Price Forecast Based on Dueling Deep Recurrent Q-network. Proceedings of the 2023 IEEE 6th International Conference on Pattern Recognition and Artificial Intelligence (PRAI), Haikou, China.10.1109/PRAI59366.2023.10332127 ↗
24. Ye, Z.J., and Björn, W.S. (2023). Human-aligned trading by imitative multi-loss reinforcement learning. Expert Syst. Appl., 234.10.1016/j.eswa.2023.120939 ↗
25. Raj, S., Ashutosh, T., Tej, B., and Uday, R. (2020, January 2–4). Real-Time Stock Market Forecasting Using Ensemble Deep Learning and Rainbow DQN. Proceedings of the 3rd International Conference on Advances in Science & Technology (ICAST), Bahir Dar, Ethiopia. EngRN: Operations Research (Topic).
26. Image smoothing based on global sparsity decomposition and a variable parameter10.1007/s41095-021-0220-1 ↗
27. Application of Wavelet Transform and Its Advantages Compared to Fourier Transform
28. Neural Networks and Wavelet De-Noising for Stock Trading and Prediction
29. Fang, Y., Fataliyev, K., Wang, L., Fu, X., and Wang, Y. (2014, January 6–11). Improving the genetic-algorithm-optimized wavelet neural network for stock market prediction. Proceedings of the 2014 International Joint Conference on Neural Networks (IJCNN), Beijing, China.10.1109/IJCNN.2014.6889969 ↗
30. Wang, J., Wang, Z., Li, J., and Wu, J. (2018, January 19–23). Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery, London, UK.10.1145/3219819.3220060 ↗
31. Bai, S., Kolter, J., and Koltun, V. (2018). An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv.
32. Jiaying, G., and Hoda, E. (2021, January 8–12). Multi-Stage Hybrid Attentive Networks for Knowledge-Driven Stock Movement Prediction. Proceedings of the International Conference on Neural Information Processing, Sanur, Bali, Indonesia.
33. Dechun, W., Tianlong, Z., Lexin, F., Caiming, Z., and Xuemei, L. (2024). MWDINet: A multilevel wavelet decomposition interaction network for stock price prediction. Expert Syst. Appl., 238.10.1016/j.eswa.2023.122091 ↗
34. Wavelet Methods for Time Series Analysis
35. Estimating daily reference evapotranspiration based on limited meteorological data using deep learning and classical machine learning methods10.1016/j.jhydrol.2020.125286 ↗
36. Hasselt, H.V., Guez, A., and Silve, D. (2016, January 12–17). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the AAAI’16: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.
37. Bellemare, M.G., Dabney, W., and Rémi, A. (2017, January 6–11). A Distributional Perspective on Reinforcement Learning. Proceedings of the International Conference on Machine Learning, Sydney, Australia.
38. Fortunato, M., Azar, M.G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., and Pietquin, O. (2017). Noisy Networks for Exploration. arXiv.
39. Kristopher, D.A., Fernando, H., Zach, H., and Richard, S. (2017). Multi-step Reinforcement Learning: A Unifying Algorithm. arXiv.