Cryptocurrency Futures Portfolio Trading System Using Reinforcement Learning
Source abstract
Abstract
This paper proposes a cryptocurrency portfolio trading system (CPTS) that optimizes trading performance in the cryptocurrency futures market by leveraging reinforcement learning and timeframe analysis. By employing the advantage actor–critic (A2C) algorithm and analysis of variance (ANOVA) portfolios are constructed over multiple timeframes. Data corresponding to the trade of 18 major cryptocurrencies on Binance Futures––between January 2022 and December 2023––are used to show that trading strategies can be effectively categorized into those with high-frequency (10, 30, and 60 min) and low-frequency (daily) timeframes. Empirical results demonstrate statistically significant differences in returns between these timeframe groups, with major cryptocurrencies (e.g., Bitcoin and Ethereum) exhibiting higher returns in high-frequency trading (16–17%) than in daily trading (6–7%) during training. Performance evaluation during the test period revealed that the low-frequency group achieved a 43.06% average return, significantly outperforming the high-frequency group (5.68%). The ANOVA results confirm that both the frequency type and portfolio selection significantly influence trading performance at the 5% significance level. This study offers a novel approach to cryptocurrency trading that considers the distinct characteristics of different timeframes. The effectiveness of combining reinforcement learning with statistical analysis for portfolio optimization in highly volatile cryptocurrency markets is demonstrated.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Blockchain Technology Applications and Security · Financial Markets and Investment Strategies · Stock Market Forecasting Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Business School, Kwangwoon University, 26 Kwangwoon-gil, Nowon-Gu, Seoul 139-701, Republic of Korea
Funding
Research Grant of Kwangwoon University
References (52)
Show source-provided references
1. Nakamoto, S. (2025, August 17). Bitcoin: A Peer-to-Peer Electronic Cash System. Bitcoin White Paper. Available online: https://bitcoin.org/bitcoin.pdf.
2. Buterin, V. (2025, August 17). A Next-Generation Smart Contract and Decentralized Application Platform. Ethereum White Paper. Available online: https://ethereum.org/en/whitepaper/.
3. Datestamping the Bitcoin and Ethereum Bubbles10.1016/j.frl.2017.12.006 ↗
4. Liang, Z., Chen, H., Zhu, J., Jiang, K., and Li, Y. (2018). Adversarial Deep Reinforcement Learning in Portfolio Management. arXiv.
5. Tether (2025, August 17). Tether: Fiat Currencies on the Bitcoin Blockchain. Tether White Paper. Available online: https://assets.ctfassets.net/vyse88cgwfbl/5UWgHMvz071t2Cq5yTw5vi/c9798ea8db99311bf90ebe0810938b01/TetherWhitePaper.pdf.
6. Can Stablecoins Foster Cryptocurrencies Adoption?
7. Bullmann, D., Klemm, J., and Pinna, A. (2019). Search for Stability in Crypto-Assets: Are Stablecoins the Solution?, European Central Bank (ECB).10.2139/ssrn.3444847 ↗
8. Bitcoin: Medium of Exchange or Speculative Assets?10.1016/j.intfin.2017.12.004 ↗
9. What Keeps Stablecoins Stable?10.1016/j.jimonfin.2022.102777 ↗
10. Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press. [2nd ed.].
11. Human-Level Control through Deep Reinforcement Learning10.1038/nature14236 ↗
12. Reinforcement Learning in Robotics: A Survey10.1177/0278364913495721 ↗
13. Mastering the Game of Go with Deep Neural Networks and Tree Search10.1038/nature16961 ↗
14. Li, Y. (2017). Deep Reinforcement Learning: An Overview. arXiv.
15. Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016). Asynchronous Methods for Deep Reinforcement Learning. arXiv.
16. Yli-Huumo, J., Ko, D., Choi, S., Park, S., and Smolander, K. (2016). Where Is Current Research on Blockchain Technology?—A Systematic Review. PLoS ONE, 11.10.1371/journal.pone.0163477 ↗
17. Carlsten, M., Kalodner, H., Weinberg, S.M., and Narayanan, A. (2016, January 24–28). On the Instability of Bitcoin without the Block Reward. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, New York, NY, USA.10.1145/2976749.2978408 ↗
18. Kroll, J.A., Davey, I.C., and Felten, E.W. (2013, January 10–11). The Economics of Bitcoin Mining, or Bitcoin in the Presence of Adversaries. Proceedings of the 12th Workshop on the Economics of Information Security (WEIS 2013), Washington, DC, USA.
19. Clark, J.A., Miller, A., Bonneau, J., Narayanan, A., and Felten, E.W. (2015, January 17–21). Research Perspectives and Challenges for Bitcoin and Cryptocurrencies. Proceedings of the IEEE Symposium on Security and Privacy, San Jose, CA, USA.
20. Wood, G. (2025, August 17). Ethereum: A Secure Decentralised Generalised Transaction Ledger. Ethereum Yellow Paper. Available online: https://ethereum.github.io/yellowpaper/paper.pdf.
21. Cryptocurrencies as a Financial Asset: A Systematic Analysis10.1016/j.irfa.2018.09.003 ↗
22. Kristoufek, L. (2015). What Are the Main Drivers of the Bitcoin Price? Evidence from Wavelet Coherence Analysis. PLoS ONE, 10.10.1371/journal.pone.0123923 ↗
23. McNally, S., Roche, J., and Caton, S. (2018, January 21–23). Predicting the Price of Bitcoin Using Machine Learning. Proceedings of the 2018 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP), Cambridge, UK.10.1109/PDP2018.2018.00060 ↗
24. Ji, S., Kim, J., and Im, H. (2019). A Comparative Study of Bitcoin Price Prediction Using Deep Learning. Mathematics, 7.10.3390/math7100898 ↗
25. Deep Direct Reinforcement Learning for Financial Signal Representation and Trading10.1109/TNNLS.2016.2522401 ↗
26. A Multi-Layer and Multi-Ensemble Stock Trader Using Deep Learning and Deep Reinforcement Learning10.1007/s10489-020-01839-5 ↗
27. Fischer, T. (2025, August 17). Reinforcement Learning in Financial Markets—A Survey. FAU Discussion Papers in Economics. Available online: https://www.econstor.eu/bitstream/10419/183139/1/1032172355.pdf.
28. Jiang, Z., and Liang, J. (2017, January 7–8). Cryptocurrency Portfolio Management with Deep Reinforcement Learning. Proceedings of the 2017 Intelligent Systems Conference (IntelliSys), London, UK.10.1109/IntelliSys.2017.8324237 ↗
29. Zhao, D., Rinaldo, A., and Brookins, C. (2019). Cryptocurrency Price Prediction and Trading Strategies Using Support Vector Machines. arXiv.
30. Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., and Meger, D. (2018, January 2–7). Deep Reinforcement Learning That Matters. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11694 ↗
31. Q-Learning
32. Sutton, R.S., McAllester, D., Singh, S., and Mansour, Y. (1999). Policy Gradient Methods for Reinforcement Learning with Function Approximation. Advances in Neural Information Processing Systems 12, MIT Press.
33. Konda, V.R., and Tsitsiklis, J.N. (2000). Actor-Critic Algorithms. Advances in Neural Information Processing Systems 13, MIT Press.
34. Pricope, T.V. (2021). Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review. arXiv.
35. Chan, E.P. (2013). Algorithmic Trading: Winning Strategies and Their Rationale, John Wiley & Sons.10.1002/9781118676998 ↗
36. Kaufman, P.J. (2013). Trading Systems and Methods, John Wiley & Sons. [5th ed.].10.1002/9781119202561 ↗
37. Pardo, R. (2008). The Evaluation and Optimization of Trading Strategies, John Wiley & Sons.
38. What Do We Know about the Profitability of Technical Analysis?10.1111/j.1467-6419.2007.00519.x ↗
39. Aldridge, I. (2013). High-Frequency Trading: A Practical Guide to Algorithmic Strategies and Trading Systems, John Wiley & Sons.10.1002/9781119203803 ↗
40. Murphy, J.J. (1999). Technical Analysis of the Financial Markets: A Comprehensive Guide to Trading Methods and Applications, Penguin.
41. Wilder, J.W. (1978). New Concepts in Technical Trading Systems, Trend Publishing Research.
42. Tsay, R.S. (2005). Analysis of Financial Time Series, John Wiley & Sons.10.1002/0471746193 ↗
43. Chincarini, L.B. (2006). Quantitative Equity Portfolio Management: An Active Approach to Portfolio Construction and Management, McGraw-Hill.
44. Empirical Asset Pricing via Machine Learning10.1093/rfs/hhaa009 ↗
45. Dixon, M.F., Halperin, I., and Bilokon, P. (2020). Machine Learning in Finance: From Theory to Practice, Springer.10.1007/978-3-030-41068-1 ↗
46. Portfolio Selection
47. De Prado, M.L. (2018). Advances in Financial Machine Learning, John Wiley & Sons.
48. Zhang, Z., Zohren, S., and Roberts, S. (2020). Deep Reinforcement Learning for Trading. arXiv.10.3905/jfds.2020.1.030 ↗
49. Informed Trading in the Bitcoin Market10.1016/j.frl.2017.11.009 ↗
50. Order Flow Analysis of Cryptocurrency Markets10.1007/s42521-019-00007-w ↗
51. Bitcoin, Gold and the Dollar––A GARCH Volatility Analysis10.1016/j.frl.2015.10.008 ↗
52. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗