Summary of Whispers That Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language Models, by Hongbang Yuan et al.
Whispers that Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language Models
by Hongbang Yuan, Pengfei Cao, Zhuoran Jin, Yubo Chen, Daojian Zeng, Kang Liu, Jun Zhao
First submitted to arxiv on: 29 Feb 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates the issue of hallucinations in Large Language Models (LLMs), specifically focusing on false premise hallucinations. It defines this phenomenon as LLMs generating text based on false premises when confronted with such questions. The authors analyze the internal working mechanism of false premise hallucination, identifying a small subset of attention heads that disrupt the knowledge extraction process. To mitigate this issue, they propose a novel method called FAITH (False premise Attention head constraining for mitigating Hallucinations), which constrains these problematic attention heads during model inference. Experimental results show that constraining just 1% of the attention heads leads to a significant performance increase of nearly 20%. This study aims to improve the reliability and accuracy of LLMs by addressing this critical limitation. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large Language Models (LLMs) can sometimes produce incorrect answers when given false information. Researchers have found that this problem is caused by a small group of attention heads in the model, which disturb the process of extracting useful knowledge from text. To fix this issue, they developed a new method called FAITH that helps the model ignore these problematic attention heads during thinking. As a result, LLMs become more accurate and reliable when answering questions. |
Keywords
» Artificial intelligence » Attention » Hallucination » Inference
