Date
AI Overview
Why it matters
Although explainable AI aims to help users act on recommendations, explanations often fail to support practical action in real-world settings, and researchers lack a systematic way to connect explanations to user actions.
Study at a glance
- Question: What information do professionals need from explainable AI, and what actions do they take with it?
- Setting and data: Online interviews with 9 doctors and 5 middle and high school computer science teachers.
- Method: Formative scenario-based design interviews and qualitative thematic analysis.
Key findings
- Participants articulated 60 distinct user actions across three broad dimensions: 13 AI interactions, 17 external physical actions, and 30 internal mental state actions.
- Mental state actions such as changing beliefs, understanding models, or setting goals were the most common, occurring 2,190 times compared to 200 AI interactions and 270 external actions.
- Users relied on 12 categories of information spanning model exposure, model creation, user environment, and user accountability, frequently seeking cross-references from outside trusted sources.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
A core assumption of Explainable AI (XAI) is that explanations are useful to users -- that is, users will do something with the explanations. Prior work, however, does not clearly connect the information provided in explanations to user actions to evaluate effectiveness. In this paper, we articulate this connection. We conducted a formative study through 14 interviews with end users in education and medicine. We contribute a catalog of information and associated actions. Our catalog maps 12 categories of information that participants described relying on to take 60 different actions. We show how AI Creators can use the catalog's specificity and breadth to articulate how they expect information in their explanations to lead to user actions and test their assumptions. We use an exemplar XAI system to illustrate this approach. We conclude by discussing how our catalog expands the design space for XAI systems to support actionability.
Research Categories
What is the application?
Who age?
Why use AI?
Publication Source
Publisher
arXiv
