Summary of Context-parametric Inversion: Why Instruction Finetuning May Not Actually Improve Context Reliance, by Sachin Goyal et al.
Context-Parametric Inversion: Why Instruction Finetuning May Not Actually Improve Context Reliance
by Sachin Goyal, Christina Baek, J. Zico Kolter, Aditi Raghunathan
First submitted to arxiv on: 14 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores an intriguing phenomenon called “context-parametric inversion” where large language models initially adapt to user context better during instruction finetuning but then gradually lose this ability. This happens while performance on standard benchmarks continues to improve. The authors observe this issue across various general-purpose instruction tuning datasets and model families, including Llama, Mistral, and Pythia. They conduct controlled studies and theoretical analysis to identify the root cause of context-parametric inversion, attributing it to examples in the finetuning data where the input context aligns with the model’s pre-existing knowledge. The authors propose some mitigation strategies, offering a starting point for addressing this issue. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine you’re teaching a computer language by giving it examples and guidance. You’d expect the computer to learn from these examples and adjust its behavior accordingly. Surprisingly, researchers found that computers actually become less good at following instructions as they learn! This happens even though their overall performance improves on standard tests. The study looked at different types of models and datasets and discovered this problem occurs across the board. The authors tried to figure out why this is happening and think it’s because the computer is relying too heavily on its initial knowledge. They suggest some ways to fix this issue, which could make computers better at learning from humans. |
Keywords
* Artificial intelligence * Instruction tuning * Llama
