An Overview of the Empirical Evaluation of Explainable AI (XAI): A Comprehensive Guideline for User-Centered Evaluation in XAI
Source abstract
Abstract
Recent advances in technology have propelled Artificial Intelligence (AI) into a crucial role in everyday life, enhancing human performance through sophisticated models and algorithms. However, the focus on predictive accuracy has often resulted in opaque black-box models that lack transparency in decision-making. To address this issue, significant efforts have been made to develop explainable AI (XAI) systems that make outcomes comprehensible to users. Various approaches, including new concepts, models, and user interfaces, aim to improve explainability, build user trust, enhance satisfaction, and increase task performance. Evaluation research has emerged to define and measure the quality of these explanations, differentiating between formal evaluation methods and empirical approaches that utilize techniques from psychology and human–computer interaction. Despite the importance of empirical studies, evaluations remain underutilized, with literature reviews indicating a lack of rigorous evaluations from the user perspective. This review aims to guide researchers and practitioners in conducting effective empirical user-centered evaluations by analyzing several studies; categorizing their objectives, scope, and evaluation metrics; and offering an orientation map for research design and metric measurement.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Explainable Artificial Intelligence (XAI) · Artificial Intelligence in Healthcare and Education · Ethics and Social Impacts of AI
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Information Systems, University of Siegen, 57072 Siegen, Germany
Bikar Metalle GmbH, 57319 Bad Berleburg, Germany
References (100+)
Show source-provided references
1. Abdul, A., Vermeulen, J., Wang, D., Lim, B.Y., and Kankanhalli, M. (2018, January 21–26). Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, in CHI ’18; Association for Computing Machinery, Montreal, QC, Canada.10.1145/3173574.3174156 ↗
2. Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy10.1080/10447318.2020.1741118 ↗
3. Doshi-Velez, F., and Kim, B. (2017). Towards A Rigorous Science of Interpretable Machine Learning. arXiv, Available online: https://api.semanticscholar.org/CorpusID:11319376.
4. Keeping the organization in the loop: A socio-technical extension of human-centered artificial intelligence10.1007/s00146-022-01391-5 ↗
5. Notions of explainability and evaluation approaches for explainable artificial intelligence10.1016/j.inffus.2021.05.009 ↗
6. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI10.1016/j.inffus.2019.12.012 ↗
7. DARPA’s Explainable Artificial Intelligence (XAI) Program
8. A systematic review and taxonomy of explanations in decision support and recommender systems10.1007/s11257-017-9195-0 ↗
9. Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., and Kagal, L. (2018, January 1–3). Explaining Explanations: An Overview of Interpretability of Machine Learning. Proceedings of the 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), Turin, Italy.10.1109/DSAA.2018.00018 ↗
10. Zhou, J., Gandomi, A.H., Chen, F., and Holzinger, A. (2021). Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics. Electronics, 10.10.3390/electronics10050593 ↗
11. Doshi-Velez, F., Kortz, M., Budish, R., Bavitz, C., Gershman, S., O’Brien, D., Scott, K., Schieber, S., Waldo, J., and Wood, A. (2017). Accountability of AI under the law: The role of explanation. arXiv.10.2139/ssrn.3064761 ↗
12. Nguyen, A., and Martínez, M.R. (2019). MonoNet: Towards Interpretable Models by Learning Monotonic Features. arXiv.
13. Rosenfeld, A. (2024, July 25). Better Metrics for Evaluating Explainable Artificial Intelligence. Available online: https://api.semanticscholar.org/CorpusID:233453690.
14. Sharp, H., Preece, J., and Rogers, Y. (2002). Interaction Design: Beyond Human-Computer Interaction, Jon Wiley & Sons. Inc.
15. Chromik, M., and Schuessler, M. (2020, January 17–20). A Taxonomy for Human Subject Evaluation of Black-Box Explanations in XAI. Proceedings of the ExSS-ATEC@IUI, Cagliari, Italy. Available online: https://api.semanticscholar.org/CorpusID:214730454.
16. Mohseni, S., Block, J.E., and Ragan, E. (2021, January 14–17). Quantitative Evaluation of Machine Learning Explanations: A Human-Grounded Benchmark. Proceedings of the 26th International Conference on Intelligent User Interfaces, in IUI ’21, College Station, TX, USA.10.1145/3397481.3450689 ↗
17. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)10.1109/ACCESS.2018.2870052 ↗
18. Anjomshoae, S., Najjar, A., Calvaresi, D., and Främling, K. (2019, January 13–17). Explainable Agents and Robots: Results from a Systematic Literature Review. Proceedings of the AAMAS ’19: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, International Foundation for Autonomous Agents and MultiAgent Systems, Montreal, QC, Canada. Available online: http://www.ifaamas.org/Proceedings/aamas2019/pdfs/p1078.pdf.
19. From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI10.1145/3583558 ↗
20. Scoping studies: Towards a methodological framework10.1080/1364557032000119616 ↗
21. Explanation in artificial intelligence: Insights from the social sciences10.1016/j.artint.2018.07.007 ↗
22. Litwin, M.S. (1995). How to Measure Survey Reliability and Validity, Sage.10.4135/9781483348957 ↗
23. Robert, F., and Carolyn, T.T. (2003). Scale Development: Theory and Applications, Sage.
24. Raykov, T., and Marcoulides, G.A. (2010). Introduction to Psychometric Theory, Routledge. [1st ed.].10.4324/9780203841624 ↗
25. Naveed, S., Kern, D.R., and Stevens, G. (2022, January 18–23). Explainable Robo-Advisors: Empirical Investigations to Specify and Evaluate a User-Centric Taxonomy of Explanations in the Financial Domain. Proceedings of the IntRS@RecSys, Seattle, WA, USA.
26. Millecamp, M., Naveed, S., Verbert, K., and Ziegler, J. (2019, January 16–19). To Explain or not to Explain: The Effects of Personal Characteristics when Explaining Feature-based Recommendations in Different Domains. Proceedings of the IntRS@RecSys, Copenaghen, Denmark. Available online: https://api.semanticscholar.org/CorpusID:203415984.
27. Naveed, S., Loepp, B., and Ziegler, J. (2020, January 12–18). On the Use of Feature-based Collaborative Explanations: An Empirical Comparison of Explanation Styles. Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, in UMAP ’20 Adjunct, Genoa Italy.10.1145/3386392.3399303 ↗
28. Naveed, S., Donkers, T., and Ziegler, J. (2018, January 8–11). Argumentation-Based Explanations in Recommender Systems: Conceptual Framework and Empirical Results. Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization, in UMAP ’18, Singapore.10.1145/3213586.3225240 ↗
29. An Empirical Study of Folk Concepts and People’s Expectations of Current and Future Artificial Intelligence10.1515/icom-2021-0009 ↗
30. Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., and Vaughan, J.W. (2020, January 25–30). Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, in CHI ’20, Honolulu, HI, USA.10.1145/3313831.3376219 ↗
31. Lai, V., Liu, H., and Tan, C. (2020, January 25–30). ‘Why is “Chicago” deceptive? ’ Towards Building Model-Driven Tutorials for Humans. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, in CHI ’20, Honolulu, HI, USA.10.1145/3313831.3376873 ↗
32. Ngo, T., Kunkel, J., and Ziegler, J. (2020, January 12–18). Exploring Mental Models for Transparent and Controllable Recommender Systems: A Qualitative Study. Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, in UMAP ’20, Genoa Italy.10.1145/3340631.3394841 ↗
33. Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., and Wong, W.-K. (2013, January 15–19). Too much, too little, or just right? Ways explanations impact end users’ mental models. Proceedings of the 2013 IEEE Symposium on Visual Languages and Human Centric Computing, San Jose, CA, USA.10.1109/VLHCC.2013.6645235 ↗
34. Sukkerd, R. (2022). Improving Transparency and Intelligibility of Multi-Objective Probabilistic Planning. [Ph.D. Thesis, Carnegie Mellon University].
35. Hoffman, R.R., Mueller, S.T., Klein, G., and Litman, J. (2019). Metrics for Explainable AI: Challenges and Prospects. arXiv.
36. Anik, A.I., and Bunt, A. (2021, January 8–12). Data-Centric Explanations: Explaining Training Data of Machine Learning Systems to Promote Transparency. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, in CHI ’21, Yokohama, Japan.10.1145/3411764.3445736 ↗
37. Hannah, D. (2022). Criteria and Metrics for the Explainability of Software, Gottfried Wilhelm Leibniz Universität.
38. Guo, L., Daly, E.M., Alkan, O., Mattetti, M., Cornec, O., and Knijnenburg, B. (2022, January 21–25). Building Trust in Interactive Machine Learning via User Contributed Interpretable Rules. Proceedings of the 27th International Conference on Intelligent User Interfaces, in IUI ’22, Helsinki, Finland.10.1145/3490099.3511111 ↗
39. Dominguez, V., Messina, P., Donoso-Guzmán, I., and Parra, D. (2019, January 16–20). The effect of explanations and algorithmic accuracy on visual recommender systems of artistic images. Proceedings of the 24th International Conference on Intelligent User Interfaces, in IUI ’19, Los Angeles, CA, USA.10.1145/3301275.3302274 ↗
40. A novel model usability evaluation framework (MUsE) for explainable artificial intelligence10.1016/j.inffus.2021.11.017 ↗
41. Millecamp, M., Htun, N.N., Conati, C., and Verbert, K. (2019, January 16–20). To explain or not to explain: The effects of personal characteristics when explaining music recommendations. Proceedings of the 24th International Conference on Intelligent User Interfaces, in IUI ’19, Los Angeles, CA, USA.10.1145/3301275.3302313 ↗
42. Buçinca, Z., Lin, P., Gajos, K.Z., and Glassman, E.L. (2020, January 17–20). Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. Proceedings of the 25th International Conference on Intelligent User Interfaces, in IUI ’20, Cagliari, Italy.10.1145/3377325.3377498 ↗
43. Cheng, H.-F., Wang, R., Zhang, Z., O’Connell, F., Gray, T., Harper, F.M., and Zhu, H. (2019, January 4–9). Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, in CHI ’19, Glasgow, UK.10.1145/3290605.3300789 ↗
44. Measuring the Quality of Explanations: The System Causability Scale (SCS)10.1007/s13218-020-00636-z ↗
45. Weina, J., and Hamarneh, G. (2023). The XAI alignment problem: Rethinking how should we evaluate human-centered AI explainability techniques. arXiv.
46. Papenmeier, A., Englebienn, G., and Seifert, C. (2019). How model accuracy and explanation fidelity influence user trust. arXiv.
47. Liao, M., and Sundar, S.S. (2021, January 8–13). How Should AI Systems Talk to Users when Collecting their Personal Information? Effects of Role Framing and Self-Referencing on Human-AI Interaction. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, in CHI ’21, Yokohama, Japan.10.1145/3411764.3445415 ↗
48. Cai, C.J., Jongejan, J., and Holbrook, J. (2019, January 17–20). The effects of example-based explanations in a machine learning interface. Proceedings of the 24th International Conference on Intelligent User Interfaces, in IUI ’19, Marina del Ray, CA, USA.10.1145/3301275.3302289 ↗
49. Evaluating XAI: A comparison of rule-based and example-based explanations10.1016/j.artint.2020.103404 ↗
50. Poursabzi-Sangdeh, F., Goldstein, D.G., Hofman, J.M., Vaughan, J.W.W., and Wallach, H. (2021, January 8–13). Manipulating and Measuring Model Interpretability. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, in CHI ’21, Yokohama, Japan.10.1145/3411764.3445315 ↗
51. Narayanan, M., Chen, E., He, J., Kim, B., Gershman, S., and Doshi-Velez, F. (2018). How do humans understand explanations from machine learning systems? An evaluation of the human-interpretability of explanation. arXiv.
52. Liu, H., Lai, V., and Tan, C. (2021). Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making. ACM Human_Computer Interaction, 5 (CSCW2), Association for Computing Machinery.10.1145/3479552 ↗
53. Schmidt, P., and Biessmann, F. (2019). Quantifying Interpretability and Trust in Machine Learning Systems. arXiv.
54. Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., and Hassner, T. (2022). HIVE: Evaluating the Human Interpretability of Visual Explanations. Computer Vision—ECCV 2022, Springer Nature.10.1007/978-3-031-20068-7 ↗
55. Rader, E., Cotter, K., and Cho, J. (2018, January 21–26). Explanations as Mechanisms for Supporting Algorithmic Transparency. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, in CHI ’18, Montreal, Canada.10.1145/3173574.3173677 ↗
56. Ooge, J., Kato, S., and Verbert, K. (2022, January 21–25). Explaining Recommendations in E-Learning: Effects on Adolescents’ Trust. Proceedings of the 27th International Conference on Intelligent User Interfaces, in IUI ’22, Helsinki, Finland.10.1145/3490099.3511140 ↗
57. Tsai, C.-H., You, Y., Gui, X., Kou, Y., and Carroll, J.M. (2021, January 8–12). Exploring and Promoting Diagnostic Transparency and Explainability in Online Symptom Checkers. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, in CHI ’21, Yokohama, Japan.10.1145/3411764.3445101 ↗
58. Guesmi, M., Chatti, M.A., Vorgerd, L., Ngo, T., Joarder, S., Ain, Q.U., and Muslim, A. (2022, January 4–7). Explaining User Models with Different Levels of Detail for Transparent Recommendation: A User Study. Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization, in UMAP ’22 Adjunct, Barcelona, Spain.10.1145/3511047.3537685 ↗
59. Rousseau, J.J., and Kapralos, B. (2023). Explaining Classifications to Non-experts: An XAI User Study of Post-Hoc Explanations for a Classifier When People Lack Expertise. Pattern Recognition, Computer Vision, and Image Processing. ICPR 2022 International Workshops and Challenges, Springer Nature.
60. Bansal, G., Wu, T., Zhou, J., Fok, R., Kamar, E., Ribeiro, M.T., and Weld, D. (2021, January 8–12). Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. Proceedings of the 2021 CHI Conference on Human. Factors in Computing Systems, in CHI ’21, Yokohama, Japan.10.1145/3411764.3445717 ↗
61. Kim, D.H., Hoque, E., and Agrawala, M. (2020, January 25–30). Answering Questions about Charts and Generating Visual Explanations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, in CHI ’20, Honolulu, HI, USA.10.1145/3313831.3376467 ↗
62. Said, A., and Komatsu, T. (2018, January 11). What Should Be in an XAI Explanation? What IFT Reveals. Proceedings of the 2018 Joint ACM IUI Workshops Co-Located with the 23rd ACM Conference on Intelligent User Interfaces (ACM IUI 2018), Tokyo, Japan. Available online: https://ceur-ws.org/Vol-2068/exss9.pdf.
63. Human-centered XAI: Developing design patterns for explanations of clinical decision support systems10.1016/j.ijhcs.2021.102684 ↗
64. Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P.S., and Vaughan, J.W. (2021, January 6–14). The Utility of Explainable AI in Ad Hoc Human-Machine Teaming. Proceedings of the Advances in Neural Information Processing Systems, online. Available online: https://dl.acm.org/doi/10.5555/3540261.3540308.
65. Does Explainable Artificial Intelligence Improve Human Decision-Making?
66. Schaffer, J., O’Donovan, J., Michaelis, J., Raglin, A., and Höllerer, T. (2019, January 16–20). I can do better than your AI: Expertise and explanations. Proceedings of the 24th International Conference on Intelligent User Interfaces, in IUI ’19, Los Angeles, CA, USA.10.1145/3301275.3302308 ↗
67. Colley, M., Eder, B., Rixen, J.O., and Rukzio, E. (2021, January 8–12). Effects of Semantic Segmentation Visualization on Trust, Situation Awareness, and Cognitive Load in Highly Automated Vehicles. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, in CHI ’21, Yokohama, Japan.10.1145/3411764.3445351 ↗
68. Zhang, Y., Liao, Q.V., and Bellamy, R.K.E. (2020, January 27–30). Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, in FAT* ’20, Barcelona, Spain.10.1145/3351095.3372852 ↗
69. Feature-Based Explanations Don’t Help People Detect Misclassifications of Online Toxicity10.1609/icwsm.v14i1.7282 ↗
70. Schoeffer, J., Kuehl, N., and Machowski, Y. (2022, January 21–24). ‘There Is Not Enough Information’: On the Effects of Explanations on Perceptions of Informational Fairness and Trustworthiness in Automated Decision-Making. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, in FAccT ’22, Seoul, Republic of Korea.10.1145/3531146.3533218 ↗
71. Kunkel, J., Donkers, T., Michael, L., Barbu, C.-M., and Ziegler, J. (2019, January 4–9). Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, in CHI ’19, Glasgow, UK.10.1145/3290605.3300717 ↗
72. Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., and Lin, H. (2020, January 6–12). How Can I Explain This to You? An Empirical Study of Deep Neural Network Explanation Methods. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, Canada. Available online: https://proceedings.neurips.cc/paper_files/paper/2020/file/2c29d89cc56cdb191c60db2f0bae796b-Paper.pdf.
73. Harrison, G., Hanson, J., Jacinto, C., Ramirez, J., and Ur, B. (2020, January 27–30). An Empirical Study on the Perceived Fairness of Realistic, Imperfect Machine Learning Models. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, in FAT* ’20, Barcelona, Spain.10.1145/3351095.3372831 ↗
74. Weitz, K., Alexander, Z., and Elisabeth, A. (2022). What do end-users really want? investigation of human-centered xai for mobile health apps. arXiv.
75. Will Humans-in-The-Loop Become Borgs? Merits and Pitfalls of Working with AI10.25300/MISQ/2021/16553 ↗
76. Evaluating the clinical utility of artificial intelligence assistance and its explanation on the glioma grading task10.1016/j.artmed.2023.102751 ↗
77. Panigutti, C., Hamon, R., Hupont, I., Llorca, D.F., Yela, D.F., Junklewitz, H., Scalzo, S., Mazzini, G., Sanchez, I., and Garrido, J.S. (2023, January 12–15). The role of explainable AI in the context of the AI Act. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, in FAccT ’23, Chicago, IL, USA.10.1145/3593013.3594069 ↗
78. Lopes, P., Silva, E., Braga, C., Oliveira, T., and Rosado, L. (2022). XAI Systems Evaluation: A Review of Human and Computer-Centred Methods. Appl. Sci., 12.10.3390/app12199423 ↗
79. Toward Human-centered XAI in Practice: A survey10.1007/s11633-022-1407-3 ↗
80. Doshi-Velez, F., and Kim, B. (2018). Considerations for evaluation and generalization in interpretable machine learning. Explainable and Interpretable Models in Computer Vision and Machine Learning, Springer.10.1007/978-3-319-98131-4_1 ↗
81. Kulesza, T., Stumpf, S., Burnett, M., and Kwan, I. (2012, January 5–10). Tell Me More? The Effects of Mental Model Soundness on Personalizing an Intelligent Agent. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, in CHI ’12, Austin, TX, USA.10.1145/2207676.2207678 ↗
82. Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
83. Rigor vs. Relevance Revisited: Response to Benbasat and Zmud10.2307/249405 ↗
84. Islam, M.R., Ahmed, M.U., Barua, S., and Begum, S. (2022). A Systematic Review of Explainable Artificial Intelligence in Terms of Different Application Domains and Tasks. Appl. Sci.10.3390/app12031353 ↗
85. Holzinger, A., Kieseberg, P., Tjoa, A.M., and Weippl, E. (2022, January 23–26). Color Shadows (Part I): Exploratory Usability Evaluation of Activation Maps in Radiological Machine Learning. Proceedings of the Machine Learning and Knowledge Extraction, Vienna, Austria.
86. Grgic-Hlaca, N., Zafar, M.B., Gummadi, K.P., and Weller, A. (2018). Beyond Distributive Fairness in Algorithmic Decision Making: Feature Selection for Procedurally Fair Learning. AAAI Conf. Artif. Intell., 32.10.1609/aaai.v32i1.11296 ↗
87. Dodge, J., Liao, Q.V., Zhang, Y., Bellamy, R.K.E., and Dugan, C. (2019, January 16–20). Explaining Models: An Empirical Study of How Explanations Impact Fairness Judgment. Proceedings of the 24th International Conference on Intelligent User Interfaces, in IUI ’19, Los Angeles, CA, USA.10.1145/3301275.3302310 ↗
88. Kern, D.-R., Dethier, E., Alizadeh, F., Stevens, G., Naveed, S., Du, D., and Shajalal, M. (2023). Peeking Inside the Schufa Blackbox: Explaining the German Housing Scoring System. arXiv.
89. Naveed, S., and Ziegler, J. (2020, January 22–26). Featuristic: An interactive hybrid system for generating explainable recommendations—Beyond system accuracy. Proceedings of the IntRS@RecSys, Rio de Janeiro, Brazil. Available online: https://api.semanticscholar.org/CorpusID:225063158.
90. Naveed, S., and Ziegler, J. (2019, January 20). Feature-Driven Interactive Recommendations and Explanations with Collaborative Filtering Approach. Proceedings of the ComplexRec@ RecSys, Copenhagen, Denmark.
91. Herlocker, J.L., Konstan, J.A., and Riedl, J. (2000, January 2–6). Explaining Collaborative Filtering Recommendations. Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, in CSCW ’00, Philadelphia, PA, USA.10.1145/358916.358995 ↗
92. Tintarev, N., and Masthoff, J. (2015). Explaining Recommendations: Design and Evaluation. Recommender Systems Handbook, Springer.10.1007/978-1-4899-7637-6_10 ↗
93. Ricci, F., Rokach, L., Shapira, B., and Kantor, P. (2011). Designing and Evaluating Explanations for Recommender Systems. Recommender Systems Handbook, Springer.10.1007/978-0-387-85820-3 ↗
94. Explaining reputation assessments10.1016/j.ijhcs.2018.10.007 ↗
95. Kouki, P., Schaffer, J., Pujara, J., O’Donovan, J., and Getoor, L. (2019, January 16–20). Personalized explanations for hybrid recommender systems. Proceedings of the 24th International Conference on Intelligent User Interfaces, in IUI ’19, Los Angeles, CA, USA.10.1145/3301275.3302306 ↗
96. Le, N.L., Abel, M.-H., and Gouspillou, P. (2023, January 1–4). Combining Embedding-Based and Semantic-Based Models for Post-Hoc Explanations in Recommender Systems. Proceedings of the 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Oahu, HI, USA.10.1109/SMC53992.2023.10394410 ↗
97. News recommender system: A review of recent progress, challenges, and opportunities10.1007/s10462-021-10043-x ↗
98. Explainable Reasoning over Knowledge Graphs for Recommendation
99. Ehsan, U., Liao, Q.V., Muller, M., Riedl, M.O., and Weisz, J.D. (2021, January 8–13). Expanding Explainability: Towards Social Transparency in AI systems. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, in CHI ’21, Yokohama, Japan.10.1145/3411764.3445188 ↗
100. Explainable Artificial Intelligence (XAI): How the Visualization of AI Predictions Affects User Cognitive Load and Confidence