Summary of Measuring User Understanding in Dialogue-based Xai Systems, by Dimitry Mindlin et al.
Measuring User Understanding in Dialogue-based XAI Systems
by Dimitry Mindlin, Amelie Sophie Robrecht, Michael Morasch, Philipp Cimiano
First submitted to arxiv on: 13 Aug 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The abstract discusses the limitations of current explainable AI (XAI) systems, which provide one-way and non-personalized explanations. To address this, dialogue-based XAI systems that adapt explanations through interaction with users have been proposed. However, there is a lack of studies evaluating user’s objective model understanding in these dialogue-based approaches. This paper fills this gap by conducting controlled experiments within a dialogue framework to measure understanding in three phases. The results quantify the level of improved understanding and reveal patterns of how high and low understanding groups differ in their interaction patterns. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research looks at how people understand AI models when they get explanations through conversations with the model. Right now, most XAI systems just give one-way explanations, which isn’t very helpful. This paper tries to solve this problem by testing whether dialogue-based XAI systems that adapt explanations can actually help people understand AI better. They do this by asking users to simulate predictions made by the model they’re learning about and measure their understanding at three different points. The results show how well users understand the model before and after interacting with it, and which groups of users learn more from this interaction. |
