Summary of Scene: Evaluating Explainable Ai Techniques Using Soft Counterfactuals, by Haoran Zheng et al.
SCENE: Evaluating Explainable AI Techniques Using Soft Counterfactuals
by Haoran Zheng, Utku Pamuksuz
First submitted to arxiv on: 8 Aug 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces SCENE (Soft Counterfactual Evaluation for Natural language Explainability), a novel evaluation method that generates Soft Counterfactual explanations in zero-shot manner. Popular XAI methods like LIME and SHAP are unstable and potentially misleading, highlighting the need for standardized evaluation approach. SCENE leverages large language models to create token-based substitutions, providing contextually appropriate and semantically meaningful Soft Counterfactuals without fine-tuning. The paper adopts Validitysoft and Csoft metrics to assess model-agnostic XAI methods in text classification tasks, shedding light on strengths and limitations of various XAI techniques. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps make AI models more transparent and accountable by creating a new way to evaluate explanations. Currently, popular methods like LIME and SHAP can be misleading or unstable, so this paper introduces SCENE, a method that uses large language models to create meaningful explanations without needing lots of training data. It also compares how well different XAI techniques work in text classification tasks. |
Keywords
» Artificial intelligence » Fine tuning » Text classification » Token » Zero shot
