Summary of On Large Language Models’ Hallucination with Regard to Known Facts, by Che Jiang et al.
On Large Language Models’ Hallucination with Regard to Known Facts
by Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou
First submitted to arxiv on: 29 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Large language models (LLMs) excel in answering factoid questions but are susceptible to hallucination. This study delves into the phenomenon of LLMs possessing correct answer knowledge yet still hallucinating from an inference dynamics perspective, a novel approach not explored before in studies on hallucinations. The investigation relies on two key ideas: identifying factual questions that query the same triplet knowledge but yield different answers and utilizing mappings from residual streams to vocabulary space to measure patterns when hallucinations occur. By analyzing the dynamic curve of output token probabilities along layer depths between correct and hallucinated cases, this study reveals distinct patterns in hallucinated cases. Leveraging these dynamics as a feature, a classifier is built with an 88% success rate for accurately detecting hallucinatory predictions. This research sheds light on understanding LLMs’ hallucinations on known facts and predicting when they are hallucinating. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study looks at why large language models (LLMs) can sometimes give correct answers, but also make up information that isn’t true. The researchers want to understand what’s going on inside the model’s thinking process. They found some interesting patterns by looking at how the model responds differently when giving correct or incorrect answers. By using these patterns as clues, they built a tool that can predict with 88% accuracy whether an LLM is making something up or not. |
Keywords
* Artificial intelligence * Hallucination * Inference * Token
