Explainable AI: roles and stakeholders, desirements and challenges
Document Type
Article
Publication Date
8-17-2023
Department
Department of Cognitive and Learning Sciences
Abstract
Introduction: The purpose of the Stakeholder Playbook is to enable the developers of explainable AI systems to take into account the different ways in which different stakeholders or role-holders need to “look inside” the AI/XAI systems. Method: We conducted structured cognitive interviews with senior and mid-career professionals who had direct experience either developing or using AI and/or autonomous systems. Results: The results show that role-holders need access to others (e.g., trusted engineers and trusted vendors) for them to be able to develop satisfying mental models of AI systems. They need to know how it fails and misleads as much as they need to know how it works. Some stakeholders need to develop an understanding that enables them to explain the AI to someone else and not just satisfy their own sense-making requirements. Only about half of our interviewees said they always wanted explanations or even needed better explanations than the ones that were provided. Based on our empirical evidence, we created a “Playbook” that lists explanation desires, explanation challenges, and explanation cautions for a variety of stakeholder groups and roles. Discussion: This and other findings seem surprising, if not paradoxical, but they can be resolved by acknowledging that different role-holders have differing skill sets and have different sense-making desires. Individuals often serve in multiple roles and, therefore, can have different immediate goals. The goal of the Playbook is to help XAI developers by guiding the development process and creating explanations that support the different roles.
Publication Title
Frontiers in Computer Science
Recommended Citation
Hoffman, R.,
Mueller, S.,
Klein, G.,
Jalaeian, M.,
&
Tate, C.
(2023).
Explainable AI: roles and stakeholders, desirements and challenges.
Frontiers in Computer Science,
5.
http://doi.org/10.3389/fcomp.2023.1117848
Retrieved from: https://digitalcommons.mtu.edu/michigantech-p2/143
Publisher's Statement
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.