Date
Publisher
arXiv
Artificial intelligence (AI) systems increasingly support decision-making across critical domains, yet current explainable AI (XAI) approaches prioritize algorithmic transparency over human comprehension. While XAI methods reveal computational processes for model validation and audit, end users require explanations integrating domain knowledge, contextual reasoning, and professional frameworks. This disconnect reveals a fundamental design challenge: existing AI explanation approaches fail to address how practitioners actually need to understand and act upon recommendations. This paper introduces Explanatory AI as a complementary paradigm where AI systems leverage generative and multimodal capabilities to serve as explanatory partners for human understanding. Unlike traditional XAI that answers "How did the algorithm decide?" for validation purposes, Explanatory AI addresses "Why does this make sense?" for practitioners making informed decisions. Through theory-informed design, we synthesize multidisciplinary perspectives on explanation from cognitive science, communication research, and education with empirical evidence from healthcare contexts and AI expert interviews. Our analysis identifies five dimensions distinguishing Explanatory AI from traditional XAI: explanatory purpose (from diagnostic to interpretive sense-making), communication mode (from static technical to dynamic narrative interaction), epistemic stance (from algorithmic correspondence to contextual plausibility), adaptivity (from uniform design to personalized accessibility), and cognitive design (from information overload to cognitively aligned delivery). We derive five meta-requirements specifying what systems must achieve and formulate ten design principles prescribing how to build them.
Study at a glance
- Question: What design principles should guide generative AI systems that help professionals understand and act on recommendations?
- Setting and data: Home care settings in Germany; interviews and observations with 6 nursing staff and 6 artificial intelligence experts.
- Method: Theory-informed design combining an integrative literature review with Rapid Contextual Design interviews and deductive qualitative analysis.
Why it matters
Current explainable AI prioritises algorithmic transparency over human comprehension. The field lacks systematic, empirically grounded design principles for generative AI systems that support practitioners' practical reasoning and decision-making.
Key findings
- Explanatory artificial intelligence is distinguished from traditional explainable artificial intelligence across five key dimensions: explanatory purpose, communication mode, epistemic stance, adaptivity, and cognitive design.
- Five meta-requirements and ten actionable design principles prescribe how systems should support interpretive sense-making, dynamic narratives, contextual plausibility, personalised adaptation, and cognitive load management.
- Health care workers prioritised practical comprehensibility and actionable domain reasoning over technical algorithmic mechanics, which they found unintelligible and unhelpful under time pressure.
Summary drafted with AI from the full paper and may contain errors. See the full paper for details.
What is the application?
Who is the user?
Who age?
Why use AI?
Study design
