journal article·2022-01-28Open access

Investigating Explainability Methods in Recurrent Neural Network Architectures for Financial Time Series Data

Warren Freeborough, Terence van Zyl

Applied Sciences· Volume 12· Issue 3

Abstract

Statistical methods were traditionally primarily used for time series forecasting. However, new hybrid methods demonstrate competitive accuracy, leading to increased machine-learning-based methodologies in the financial sector. However, very little development has been seen in explainable AI (XAI) for financial time series prediction, with a growing mandate for explainable systems. This study aims to determine if the existing XAI methodology is transferable to the context of financial time series prediction. Four popular methods, namely, ablation, permutation, added noise, and integrated gradients, were applied to a recurrent neural network (RNN), long short-term memory (LSTM), and a gated recurrent unit (GRU) network trained on S&P 500 stocks data to determine the importance of features, individual data points, and specific cells in each architecture. The explainability analysis revealed that GRU displayed the most significant ability to retain long-term information, while the LSTM disregarded most of the given input and instead showed the most notable granularity to the considered inputs. Lastly, the RNN displayed features indicative of no long-term memory retention. The applied XAI methods produced complementary results, reinforcing paradigms on significant differences in how different architectures predict. The results show that these methods are transferable in the financial forecasting sector, but a more sophisticated hybrid prediction system requires further confirmation.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Stock Market Forecasting Methods · Explainable Artificial Intelligence (XAI) · Forecasting Techniques and Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Computer Science and Applied Mathematics, University of the Witwatersrand, Johannesburg 2050, South Africa

DSI-NICIS National e-Science Postgraduate Teaching and Training Platform (NEPTTP), University of the Witwatersrand, Johannesburg 2050, South Africa

Institute for Intelligent Systems, University of Johannesburg, Johannesburg 2092, South Africa

References (45)

Show source-provided references
1. A survey on explainable artificial intelligence (xai): Toward medical xai10.1109/TNNLS.2020.3027314 ↗
2. Rudin, C. (2014, January 24–27). Algorithms for interpretable machine learning. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.10.1145/2623330.2630823 ↗
3. van Zyl, T.L. (2014). Machine Learning on Geospatial Big Data. Big Data: Techniques and Technologies in Geoinformatics, CRC Press.
4. Explainable artificial intelligence: An analytical review10.1002/widm.1424 ↗
5. Addressing bias in artificial intelligence in health care10.1001/jama.2019.18058 ↗
6. Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., and Elhadad, N. (2015, January 10–13). Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia.10.1145/2783258.2788613 ↗
7. European Union regulations on algorithmic decision-making and a “right to explanation”
8. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., and Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE, 10.10.1371/journal.pone.0130140 ↗
9. Dieber, J., and Kirrane, S. (2020). Why model why? Assessing the strengths and limitations of LIME. arXiv.
10. Lundberg, S.M., and Lee, S.I. (2017, January 4–9). A unified approach to interpreting model predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
11. Riberio, M.T., Singh, S., and Guestrin, C. (2016, January 13–17). Why Should I Trust You? Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939778 ↗
12. A historical perspective of explainable Artificial Intelligence10.1002/widm.1391 ↗
13. Fauvel, K., Lin, T., Masson, V., Fromont, É., and Termier, A. (2021). XCM: An Explainable Convolutional Neural Network for Multivariate Time Series Classification. Mathematics, 9.10.3390/math9233137 ↗
14. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22–29). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.74 ↗
15. Viton, F., Elbattah, M., Guérin, J.L., and Dequen, G. (December, January 30). Heatmaps for Visual Explainability of CNN-Based Predictions for Multivariate Time Series with Application to Healthcare. Proceedings of the 2020 IEEE International Conference on Healthcare Informatics (ICHI), Oldenburg, Germany.10.1109/ICHI48887.2020.9374393 ↗
16. Meyes, R., Lu, M., de Puiseau, C.W., and Meisen, T. (2019). Ablation studies in artificial neural networks. arXiv.
17. Covert, I., Lundberg, S., and Lee, S.I. (2020). Understanding global feature contributions with additive importance measures. arXiv.
18. Sundararajan, M., Taly, A., and Yan, Q. (2017, January 6–11). Axiomatic attribution for deep networks. Proceedings of the International Conference on Machine Learning, Sydney, NSW, Australia.
19. Kopitar, L., Cilar, L., Kocbek, P., and Stiglic, G. (2019). Local vs. Global Interpretability of Machine Learning Models in Type 2 Diabetes Mellitus Screening. Artificial Intelligence in Medicine: Knowledge Representation and Transparent and Explainable Systems, Springer.10.1007/978-3-030-37446-4_9 ↗
20. Kenny, E.M., Delaney, E.D., Greene, D., and Keane, M.T. (2021, January 10–15). Post-hoc explanation options for XAI in deep learning: The Insight centre for data analytics perspective. Proceedings of the International Conference on Pattern Recognition, Virtual Event.10.1007/978-3-030-68796-0_2 ↗
21. A state space framework for automatic forecasting using exponential smoothing methods10.1016/S0169-2070(01)00110-8 ↗
22. Automatic time series forecasting: The forecast package for R10.18637/jss.v027.i03 ↗
23. Nguyen, T.T., Le Nguyen, T., and Ifrim, G. (2020). A Model-Agnostic Approach to Quantifying the Informativeness of Explanation Methods for Time Series Classification. Lecture Notes in Computer Science, Proceedings of the International Workshop on Advanced Analytics and Learning on Temporal Data, Ghent, Belgium, 18 September 2020, Springer.10.1007/978-3-030-65742-0_6 ↗
24. Delaney, E., Greene, D., and Keane, M.T. (2021). Instance-based counterfactual explanations for time series classification. Lecture Notes in Computer Science, Proceedings of the International Conference on Case-Based Reasoning, Salamanca, Spain, 13–16 September 2021, Springer.10.1007/978-3-030-86957-1_3 ↗
25. Bailey, D.H., Borwein, J., Lopez de Prado, M., Salehipour, A., and Zhu, Q.J. (2016). Backtest overfitting in financial markets. Automated Trader, 1–8. Available online: https://www.davidhbailey.com/dhbpapers/overfit-tools-at.pdf.
26. Financial time series forecasting with deep learning: A systematic literature review: 2005–201910.1016/j.asoc.2020.106181 ↗
27. Laher, S., Paskaramoorthy, A., and Van Zyl, T.L. (2021, January 1–4). Deep Learning for Financial Time Series Forecast Fusion and Optimal Portfolio Rebalancing. Proceedings of the 2021 IEEE 24th International Conference on Information Fusion (FUSION), Sun City, South Africa.10.23919/FUSION49465.2021.9626945 ↗
28. The M4 Competition: 100,000 time series and 61 forecasting methods10.1016/j.ijforecast.2019.04.014 ↗
29. Mathonsi, T., and van Zyl, T.L. (2020, January 14–15). Prediction Interval Construction for Multivariate Point Forecasts Using Deep Learning. Proceedings of the 2020 7th International Conference on Soft Computing & Machine Intelligence (ISCMI), Stockholm, Sweden.10.1109/ISCMI51676.2020.9311603 ↗
30. The M5 accuracy competition: Results, findings and conclusions
31. A Statistics and Deep Learning Hybrid Method for Multivariate Time Series Forecasting and Mortality Modeling10.3390/forecast4010001 ↗
32. Testing for unit roots in autoregressive-moving average models of unknown order10.1093/biomet/71.3.599 ↗
33. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
34. Dropout: A simple way to prevent neural networks from overfitting
35. Pytorch: An imperative style, high-performance deep learning library
36. Seabold, S., and Perktold, J. (July, January 28). Statsmodels: Econometric and statistical modeling with python. Proceedings of the 9th Python in Science Conference, Austin, TX, USA.
37. SciPy 1.0: Fundamental algorithms for scientific computing in Python10.1038/s41592-019-0686-2 ↗
38. Kokhlikyan, N., Miglani, V., Martin, M., Wang, E., Alsallakh, B., Reynolds, J., Melnikov, A., Kliushkina, N., Araya, C., and Yan, S. (2020). Captum: A unified and generic model interpretability library for pytorch. arXiv.
39. Matplotlib: A 2D graphics environment10.1109/MCSE.2007.55 ↗
40. Seaborn: Statistical data visualization10.21105/joss.03021 ↗
41. LSTM recurrent networks learn simple context-free and context-sensitive languages10.1109/72.963769 ↗
42. Trinh, T., Dai, A., Luong, T., and Le, Q. (May, January 30). Learning longer-term dependencies in rnns with auxiliary losses. Proceedings of the International Conference on Machine Learning, Vancouver, BC, Canada.
43. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv.
44. CNNpred: CNN-based stock market prediction using a diverse set of variables10.1016/j.eswa.2019.03.029 ↗
45. A CNN–LSTM model for gold price time-series forecasting10.1007/s00521-020-04867-x ↗