Summary of When Context Leads but Parametric Memory Follows in Large Language Models, by Yufei Tao et al.
When Context Leads but Parametric Memory Follows in Large Language Models
by Yufei Tao, Adam Hiatt, Erik Haake, Antonie J. Jetter, Ameeta Agrawal
First submitted to arxiv on: 13 Sep 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel study explores how nine large language models allocate knowledge when answering open-ended questions in scenarios where the provided information is consistent with their parametric knowledge. The researchers introduce a new dataset, WikiAtomic, and analyze how the models prioritize and utilize contextual and parametric knowledge across varying context sizes. They also investigate the tendency of the models to hallucinate under different context conditions. The findings reveal consistent patterns across models, including a reliance on both contextual and parametric knowledge, and a decrease in hallucinations with increasing context size. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large language models are getting smarter at using many different sources of information. A new study looked at how nine of these models use the information they’re given to answer questions. The researchers made up a new dataset called WikiAtomic and tested the models’ ability to use this information in different situations. They found that the models rely on both the context (around 70%) and their own knowledge (around 30%) to get the right answers, and that they tend to make fewer mistakes as they’re given more context. |
