From SHAP Plots to Stories: Making Attributions Human

1. The Problem with SHAP at Face Value SHAP (SHapley Additive exPlanations) has emerged as a go-to tool for interpreting model predictions. But while the visuals are elegant for data scientists, they often fall flat for business users. Color gradients, feature importance bars, and log odds don’t tell a story. They merely suggest where one might exist. This creates a disconnect between insight and impact.

2. Why Storytelling Matters in Explainability Decision-makers rarely need every detail. They need meaning. A SHAP plot might show that 'income' has a high positive impact on a loan approval, but what does that mean in a real-world context? A story might read: "The model leaned toward approval because the applicant's income level indicated strong repayment capacity, even though credit history was average."

3. Design Patterns for Narrative Attribution

• The Causal Frame: Describe what caused the prediction in a way that mirrors human reasoning.
• The Comparison Frame: Contrast the prediction with a near-miss or a counterfactual.
• The Traceable Frame: Link input factors to outcomes in sequential order, like a decision tree.

4. Tools That Bridge the Gap Using LLMs (like GPT-4) to auto-generate textual summaries of SHAP attributions is gaining momentum. Coupled with visualization overlays, this allows a two-layer explanation: one for human readability, one for model traceability.

5. Regulatory Alignment As AI regulation intensifies in regions like the EU and Canada, explainability must shift from technical defense to communicative clarity. Model cards and decision explanations should be accessible and auditable, not just logged.

6. From Output to Understanding The final mile in explainability is empathy. SHAP plots point to patterns; stories provide understanding. When stakeholders feel the reasoning behind predictions, they're more likely to engage, trust, and iterate responsibly.

1. The Problem with SHAP at Face Value

• Lundberg, S.M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. NeurIPS 2017.

This foundational paper introduces SHAP, focusing on its mathematical foundations and visual interpretability.

• Molnar, C. (2022). Interpretable Machine Learning. Book link

Critically discusses SHAP and the challenges of interpreting its outputs beyond the data science community.

📚 2. Why Storytelling Matters in Explainability

• Krause, J., Perer, A., & Ng, K. (2016). Interacting with Predictions: Visual Inspection of Black-box Machine Learning Models. CHI 2016.

Highlights how visual explanations alone are not sufficient; storytelling enhances human trust.

• Microsoft Research. (2022). Human-centered AI: Stories, not scores.

Research pushing for explainability grounded in user understanding and domain context.

3. Design Patterns for Narrative Attribution

• Weld, D.S., & Bansal, G. (2019). The Challenge of Crafting Intelligible Intelligence. Communications of the ACM.

Suggests structured templates and storytelling frames to explain model reasoning.

• Liao, Q. V., et al. (2020). Questioning the AI: Informing Design Practices for Explainable AI User Experiences. CHI 2020.

Introduces frameworks like comparison and causal attribution for UX in XAI.

4. Tools That Bridge the Gap

• OpenAI. (2023). Explaining SHAP with GPT-4.

Demos and guides showing how LLMs can summarize SHAP outputs meaningfully.

• Slack et al. (2021). Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods.

Underlines why narrative-level validation is needed, beyond raw plots.

5. Regulatory Alignment

• European Commission. (2024). EU AI Act Final Text – Article 13, Transparency Obligations.

Requires AI outputs to be interpretable by humans, especially in high-risk systems.

• Canadian Government. (2023). Directive on Automated Decision-Making.

Emphasizes explainability, audit trails, and the communication of reasoning.

6. From Output to Understanding

• IBM Research. (2022). From Explanations to Understandability in AI.

Argues that empathy and relatability are keys to successful AI communication.

• Lipton, Z.C. (2018). The Mythos of Model Interpretability. CACM.

A seminal critique on the limits of interpretation that lack human-centered framing.