Reinforcement Learning for Options Trading
Source abstract
Abstract
Reinforcement learning has been applied to various types of financial assets trading, such as stocks, futures, and cryptocurrencies. Options, as a novel kind of derivative, have their characteristics. Because there are too many option contracts for one underlying asset and their price behavior is different. Besides, the validity period of an option contract is relatively short. To apply reinforcement learning to options trading, we propose the options trading reinforcement learning (OTRL) framework. We use options’ underlying asset data to train the reinforcement learning model. Candle data in different time intervals are utilized, respectively. The protective closing strategy is added to the model to prevent unbearable losses. Our experiments demonstrate that the most stable algorithm for obtaining high returns is proximal policy optimization (PPO) with the protective closing strategy. The deep Q network (DQN) can exceed the buy and hold strategy in options trading, as can soft actor critic (SAC). The OTRL framework is verified effectively.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Stock Market Forecasting Methods · Advanced Bandit Algorithms Research · Reinforcement Learning in Robotics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing 100876, China
Key Laboratory of Trustworthy Distributed Computing and Service, Ministry of Education, Beijing 100876, China
Funding
National Natural Science Foundation of China
References (34)
Show source-provided references
1. Meng, T.L., and Khushi, M. (2019). Reinforcement learning in financial markets. Data, 4.10.3390/data4030110 ↗
2. Time-driven feature-aware jointly deep reinforcement learning for financial signal representation and algorithmic trading10.1016/j.eswa.2019.112872 ↗
3. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv.
4. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion
5. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
6. Deep robust reinforcement learning for practical algorithmic trading10.1109/ACCESS.2019.2932789 ↗
7. Application of deep reinforcement learning in stock trading strategies and stock forecasting10.1007/s00607-019-00773-w ↗
8. Bisht, K., and Kumar, A. (2020, January 1–3). Deep Reinforcement Learning based Multi-Objective Systems for Financial Trading. Proceedings of the 2020 5th IEEE International Conference on Recent Advances and Innovations in Engineering (ICRAIE), Jaipur, India.10.1109/ICRAIE51050.2020.9358319 ↗
9. Deep reinforcement learning for trading10.3905/jfds.2020.1.030 ↗
10. Si, W., Li, J., Ding, P., and Rao, R. (2017, January 9–10). A multi-objective deep reinforcement learning approach for stock index future’s intraday trading. Proceedings of the 2017 10th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China.10.1109/ISCID.2017.210 ↗
11. Lucarelli, G., and Borrotti, M. (2019, January 24–26). A deep reinforcement learning approach for automated cryptocurrency trading. Proceedings of the IFIP International Conference on Artificial Intelligence Applications and Innovations, Hersonissos, Crete, Greece.10.1007/978-3-030-19823-7_20 ↗
12. A novel approach of option portfolio construction using the Kelly criterion10.1109/ACCESS.2018.2869282 ↗
13. A markowitz portfolio approach to options trading10.1109/TSP.2018.2849733 ↗
14. Hong, M., Chang, T.H., Wang, X., Razaviyayn, M., Ma, S., and Luo, Z.Q. (2014). A block successive upper bound minimization method of multipliers for linearly constrained convex optimization. arXiv.
15. Volatility Forecast Incorporating Investors’ Sentiment and its Application in Options Trading Strategies: A Behavioural Finance Approach at Nifty 50 Index10.1177/0972262920914117 ↗
16. Wu, J.M.T., Wu, M.E., Hung, P.J., Hassan, M.M., and Fortino, G. (2020). Convert index trading to option strategies via LSTM architecture. Neural Comput. Appl., 1–18.10.1007/s00521-020-05377-6 ↗
17. Voronoi-based multi-robot autonomous exploration in unknown environments via deep reinforcement learning10.1109/TVT.2020.3034800 ↗
18. Reinforcement learning: A survey10.1613/jair.301 ↗
19. An application of deep reinforcement learning to algorithmic trading10.1016/j.eswa.2021.114632 ↗
20. Data augmentation for deep neural network acoustic modeling10.1109/TASLP.2015.2438544 ↗
21. A survey on image data augmentation for deep learning10.1186/s40537-019-0197-0 ↗
22. Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep Learning, MIT Press.
23. Fons, E., Dawson, P., Zeng, X.j., Keane, J., and Iosifidis, A. (2020). Evaluating data augmentation for financial time series classification. arXiv.
24. Enhancing stock price trend prediction via a time-sensitive data augmentation method10.1155/2020/6737951 ↗
25. Mallat, S.G. (2009). A theory for multiresolution signal decomposition: The wavelet representation. Fundamental Papers in Wavelet Theory, Princeton University Press.10.1515/9781400827268.494 ↗
26. Le Guennec, A., Malinowski, S., and Tavenard, R. (2016, January 20). Data augmentation for time series classification using convolutional neural networks. Proceedings of the ECML/PKDD Workshop on Advanced Analytics and Learning on Temporal Data, Würzburg, Germany.
27. Um, T.T., Pfister, F.M., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., and Kulić, D. (2017, January 13–17). Data augmentation of wearable sensor data for parkinson’s disease monitoring using convolutional neural networks. Proceedings of the 19th ACM International Conference on Multimodal Interaction, Glasgow, UK.10.1145/3136755.3136817 ↗
28. Kamycki, K., Kapuscinski, T., and Oszust, M. (2020). Data augmentation with suboptimal warping for time-series classification. Sensors, 20.10.3390/s20010098 ↗
29. Yuan, Y., Wen, W., and Yang, J. (2020). Using Data Augmentation Based Reinforcement Learning for Daily Stock Trading. Electronics, 9.10.3390/electronics9091384 ↗
30. Lapan, M. (2018). Deep Reinforcement Learning Hands-On: Apply Modern RL Methods, with Deep Q-Networks, Value Iteration, Policy Gradients, TRPO, AlphaGo Zero and More, Packt Publishing Ltd.
31. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
32. Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015, January 7–9). Trust region policy optimization. Proceedings of the International Conference on Machine Learning, PMLR, Lille, France.
33. Christodoulou, P. (2019). Soft actor-critic for discrete action settings. arXiv.
34. Evolutionary ORB-based model with protective closing strategies10.1016/j.knosys.2021.106769 ↗