Summary of Revealing the Parametric Knowledge Of Language Models: a Unified Framework For Attribution Methods, by Haeun Yu et al.
Revealing the Parametric Knowledge of Language Models: A Unified Framework for Attribution Methods
by Haeun Yu, Pepa Atanasova, Isabelle Augenstein
First submitted to arxiv on: 29 Apr 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper investigates how Language Models (LMs) store and utilize their training-acquired knowledge, which is embedded within their weights. The study highlights the challenges in understanding a model’s inner workings and updating or correcting this knowledge without retraining. To tackle these issues, Instance Attribution (IA) and Neuron Attribution (NA) are employed to unveil what knowledge is stored and its association with specific model components. The paper proposes a novel evaluation framework to quantify and compare the knowledge revealed by IA and NA, introducing NA-Instances and IA-Neurons as attribution methods. Faithfulness tests are also proposed to evaluate the comprehensiveness of explanations provided by both methods. Experimental results demonstrate that NA generally reveals more diverse information, while IA provides unique insights into LMs’ parametric knowledge. The study suggests a synergistic approach combining IA and NA for a holistic understanding of an LM’s parametric knowledge. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us understand how language models learn from their training data. Right now, it’s hard to know what specific parts of the model are responsible for its intelligence. Two methods called Instance Attribution (IA) and Neuron Attribution (NA) can help with this. The study compares these two methods and shows that NA is better at revealing diverse information, while IA provides unique insights. By combining both methods, we might get a more complete understanding of how language models work. |
