Explaining Deep Learning Models for Tabular Data Using Layer-Wise Relevance Propagation
Source abstract
Abstract
Trust and credibility in machine learning models are bolstered by the ability of a model to explain its decisions. While explainability of deep learning models is a well-known challenge, a further challenge is clarity of the explanation itself for relevant stakeholders of the model. Layer-wise Relevance Propagation (LRP), an established explainability technique developed for deep models in computer vision, provides intuitive human-readable heat maps of input images. We present the novel application of LRP with tabular datasets containing mixed data (categorical and numerical) using a deep neural network (1D-CNN), for Credit Card Fraud detection and Telecom Customer Churn prediction use cases. We show how LRP is more effective than traditional explainability concepts of Local Interpretable Model-agnostic Explanations (LIME) and Shapley Additive Explanations (SHAP) for explainability. This effectiveness is both local to a sample level and holistic over the whole testing set. We also discuss the significant computational time advantage of LRP (1–2 s) over LIME (22 s) and SHAP (108 s) on the same laptop, and thus its potential for real time application scenarios. In addition, our validation of LRP has highlighted features for enhancing model performance, thus opening up a new area of research of using XAI as an approach for feature subset selection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Explainable Artificial Intelligence (XAI) · Adversarial Robustness in Machine Learning · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
CeADAR Irelands Center for Applied AI, University College Dublin, D04V2N9 Dublin, Ireland
CeADAR Irelands Center for Applied AI, Technological University Dublin, D07ADY7 Dublin, Ireland
Funding
CeADAR Irelands Centre for Applied AI
References (63)
Show source-provided references
1. DARPA’s Explainable Artificial Intelligence (XAI) Program
2. ImageNet Classification with Deep Convolutional Neural Networks
3. Kolesnikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., and Houlsby, N. (2019). Large Scale Learning of General Visual Representations for Transfer. arXiv.
4. Xie, S., Girshick, R., Dollar, P., Tu, Z., and He, K. (2017, January 21–26). Aggregated Residual Transformations for Deep Neural Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.634 ↗
5. Liu, Y., Wang, Y., Wang, S., Liang, T., Zhao, Q., Tang, Z., and Ling, H. (2019). CBNet: A Novel Composite Backbone Network Architecture for Object Detection. arXiv.10.1609/aaai.v34i07.6834 ↗
6. Howard, A., Sandler, M., Chu, G., Chen, L.C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., and Vasudevan, V. (2019). Searching for MobileNetV3. arXiv.10.1109/ICCV.2019.00140 ↗
7. Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., and Yan, J. (2018). SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks. arXiv.10.1109/CVPR.2019.00441 ↗
8. Mastering the game of Go with deep neural networks and tree search10.1038/nature16961 ↗
9. Liu, J., Chao, F., Lin, Y.C., and Lin, C.M. (2019). Stock Prices Prediction using Deep Learning Models. arXiv.
10. Performance enhancing techniques for deep learning models in time series forecasting10.1016/j.engappai.2019.07.011 ↗
11. Gasparin, A., Lukovic, S., and Alippi, C. (2019). Deep Learning for Time Series Forecasting: The Electric Load Case. arXiv.
12. Artificial intelligence–based breast cancer nodal metastasis detection insights into the black box for pathologists10.5858/arpa.2018-0147-OA ↗
13. How to Explain Individual Classification Decisions
14. Automated decision-making in the EU Member States: The right to explanation and other “suitable safeguards” in the national legislations10.1016/j.clsr.2019.05.002 ↗
15. Tjoa, E., and Guan, C. (2019). A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI. arXiv.
16. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22–29). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.74 ↗
17. Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I.D., and Gebru, T. (2019, January 29–31). Model Cards for Model Reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency—FAT* ’19, Atlanta, GA, USA.10.1145/3287560.3287596 ↗
18. A survey of methods for explaining black box models10.1145/3236009 ↗
19. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI10.1016/j.inffus.2019.12.012 ↗
20. Learning Important Features Through Propagating Activation Differences
21. Guo, W., Mu, D., Xu, J., Su, P., Wang, G., and Xing, X. (2018, January 15–19). LEMNA: Explaining Deep Learning Based Security Applications. Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, Toronto, ON, Canada.10.1145/3243734.3243792 ↗
22. Plumb, G., Molitor, D., and Talwalkar, A. (2018). Supervised Local Modeling for Interpretability. arXiv.
23. Guidotti, R., Monreale, A., Ruggieri, S., Pedreschi, D., Turini, F., and Giannotti, F. (2018). Local Rule-Based Explanations of Black Box Decision Systems. arXiv.
24. Chen, J., Song, L., Wainwright, M., and Jordan, M. (2018, January 10–15). Learning to explain: An information-theoretic perspective on model interpretation. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
25. Lundberg, S.M., Erion, G., Chen, H., DeGrave, A., Prutkin, J.M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., and Lee, S.I. (2019). Explainable AI for Trees: From Local Explanations to Global Understanding. arXiv.
26. Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., and Kagal, L. (2018, January 1–3). Explaining Explanations: An Overview of Interpretability of Machine Learning. Proceedings of the 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), Turin, Italy.10.1109/DSAA.2018.00018 ↗
27. El-assady, M., Jentner, W., Kehlbeck, R., and Schlegel, U. (2019, January 4). Towards XAI: Structuring the Processes of Explanations. Proceedings of the ACM Workshop on Human-Centered Machine Learning, Glasgow, UK.
28. Clinical applications of machine learning algorithms: Beyond the black box10.1136/bmj.l886 ↗
29. Mittelstadt, B., Russell, C., and Wachter, S. (2019, January 29–31). Explaining explanations in AI. Proceedings of the 2019 Conference on Fairness, Accountability, and Transparency, Atlanta, GA, USA.10.1145/3287560.3287574 ↗
30. Samek, W., and Müller, K.R. (2019). Towards Explainable Artificial Intelligence. Lecture Notes in Computer Science, Proceedings of the Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer.10.1007/978-3-030-28954-6 ↗
31. Unmasking Clever Hans predictors and assessing what machines really learn10.1038/s41467-019-08987-4 ↗
32. Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13–17). “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939778 ↗
33. Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems 30, Curran Associates, Inc.
34. Tian, Y., and Liu, G. (2020, January 18–23). MANE: Model-Agnostic Non-linear Explanations for Deep Learning Model. Proceedings of the 2020 IEEE World Congress on Services (SERVICES), Beijing, China.10.1109/SERVICES48979.2020.00021 ↗
35. Simonyan, K., Vedaldi, A., and Zisserman, A. (2014, January 14–16). Deep inside convolutional networks: Visualising image classification models and saliency maps. Proceedings of the 2nd International Conference on Learning Representations, ICLR 2014-Workshop Track Proceedings, Banff, AB, Canada.
36. Arras, L., Montavon, G., Müller, K.R., and Samek, W. (2018). Explaining Recurrent Neural Network Predictions in Sentiment Analysis. arXiv.10.18653/v1/W17-5221 ↗
37. Zeiler, M.D., Krishnan, D., Taylor, G.W., and Fergus, R. (2010, January 13–18). Deconvolutional networks. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5539957 ↗
38. Zeiler, M.D., Taylor, G.W., and Fergus, R. (2011, January 6–13). Adaptive deconvolutional networks for mid and high level feature learning. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.10.1109/ICCV.2011.6126474 ↗
39. Zeiler, M.D., and Fergus, R. (2014). Visualizing and understanding convolutional networks. Lecture Notes in Computer Science, Proceedings of the European Conference on Computer Vision, Zurich, Switzerland, 6–12 September 2014, Springer.10.1007/978-3-319-10590-1_53 ↗
40. Zhang, Q., Wu, Y.N., and Zhu, S. (2018, January 18–20). Interpretable Convolutional Neural Networks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00920 ↗
41. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W., and Suárez, O.D. (2015). On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation. PLoS ONE, 10.10.1371/journal.pone.0130140 ↗
42. TSViz: Demystification of deep learning models for time-series analysis10.1109/ACCESS.2019.2912823 ↗
43. Bankruptcy prediction by generalized additive models10.1002/asmb.658 ↗
44. Calabrese, R. (2012). Estimating Bank Loans Loss Given Default by Generalized Additive Models, University of Milano-Bicocca.
45. New approaches to regression by generalized additive models and continuous optimization for modern applications in finance, science and technology10.1080/02331930701618740 ↗
46. Bussmann, N., Giudici, P., Marinelli, D., and Papenbrock, J. (2020). Explainable AI in Credit Risk Management. SSRN Electron. J.10.2139/ssrn.3506274 ↗
47. Decision support from financial disclosures with deep neural networks and transfer learning10.1016/j.dss.2017.10.001 ↗
48. Opening the Black Box of Financial AI with CLEAR-Trade: A CLass-Enhanced Attentive Response Approach for Explaining and Visualizing Deep Learning-Driven Stock Market Prediction
49. Poerner, N., Schütze, H., and Roth, B. (2018, January 15–20). Evaluating neural network explanation methods using hybrid documents and morphosyntactic agreement. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia.10.18653/v1/P18-1032 ↗
50. Gender Classification from Facial Images Using Texture Descriptors
51. Nguyen, T., Park, E.A., Han, J., Park, D.C., and Min, S.Y. (2014). Object detection using scale invariant feature transform. Genetic and Evolutionary Computing, Springer.10.1007/978-3-319-01796-9_7 ↗
52. Bandi, A. (2021, December 15). Telecom Churn Prediction dataset. Available online: https://www.kaggle.com/bandiatindra/telecom-churn-prediction/data.
53. ULB, Machine Learning Group (2021, December 15). Credit Card Fraud Detection Dataset. Available online: https://www.kaggle.com/mlg-ulb/creditcardfraud.
54. SMOTE: Synthetic minority over-sampling technique10.1613/jair.953 ↗
55. Ullah, I., and Petrosino, A. (2016, January 24–29). About pyramid structure in convolutional neural networks. Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada.10.1109/IJCNN.2016.7727350 ↗
56. Bandi, A. (2021, December 15). Telecom Churn Prediction. Available online: https://www.kaggle.com/bandiatindra/telecom-churn-prediction#EDA-and-Prediction.
57. Bachmann, M.J. (2021, December 15). Credit Fraud || Dealing with Imbalanced Datasets. Available online: https://www.kaggle.com/janiobachmann/credit-fraud-dealing-with-imbalanced-datasets.
58. Arga, P. (2021, December 15). Credit Card Fraud Detection. Available online: https://www.kaggle.com/joparga3/in-depth-skewed-data-classif-93-recall-acc-now.
59. Sanagapati, P. (2020, August 25). Anomaly Detection—Credit Card Fraud Analysis. Available online: https://www.kaggle.com/pavansanagapati/anomaly-detection-credit-card-fraud-analysis.
60. Media, V. (2021, December 15). 250 MB Broadband. Available online: https://www.virginmedia.ie/broadband/buy-a-broadband-package/250-mb-broadband/.
61. Mckeever, S., Ullah, I., and Rios, A. (2021, January 10–15). XPlainIT—A demonstrator for explaining deep learning models trained over structured data. Proceedings of the Demo Session of Machine Learning and Multimedia, at 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.
62. Raj, P. (2020, August 25). Telecom Churn Prediction. Available online: https://www.kaggle.com/pavanraj159/telecom-customer-churn-prediction/notebook.
63. Mohammad, N.I., Ismail, S.A., Kama, M.N., Yusop, O.M., and Azmi, A. (2019, January 26–28). Customer Churn Prediction In Telecommunication Industry Using Machine Learning Classifiers. Proceedings of the 3rd International Conference on Vision, Image and Signal Processing, Vancouver, BC, Canada.10.1145/3387168.3387219 ↗