Summary of Open Domain Question Answering with Conflicting Contexts, by Siyi Liu et al.
Open Domain Question Answering with Conflicting Contexts
by Siyi Liu, Qiang Ning, Kishaloy Halder, Wei Xiao, Zheng Qi, Phu Mon Htut, Yi Zhang, Neha Anna John, Bonan Min, Yassine Benajiba, Dan Roth
First submitted to arxiv on: 16 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates open domain question answering systems’ reliance on large text collections and how they handle conflicting information. It introduces a new dataset, Question Answering with Conflicting Contexts (QACC), which reveals that 25% of questions can lead to contradictory retrieved contexts. The authors evaluate three powerful Large Language Models (LLMs) using QACC and show their limitations in addressing questions with conflicting information. To better understand human reasoning through these conflicts, the annotators provide explanations for correct answer selections. By fine-tuning LLMs to explain their answers, the researchers demonstrate how this can guide them through reasoning processes. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about a new way to help computers answer questions by looking at lots of text on the internet. Sometimes, this text has conflicting information, which means it’s not always true. The authors created a special dataset to test how well these computer programs do when they come across this kind of information. They found that many programs struggle with this problem and can give wrong answers because of it. To make things better, the researchers asked people to explain why they chose certain answers as correct. This helped them see what the people were thinking and how they made decisions. |
Keywords
» Artificial intelligence » Fine tuning » Question answering
