Summary of Final-model-only Data Attribution with a Unifying View Of Gradient-based Methods, by Dennis Wei et al.
Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods
by Dennis Wei, Inkit Padhi, Soumya Ghosh, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Maria Chang
First submitted to arxiv on: 5 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper focuses on Training Data Attribution (TDA), a task that involves attributing model behavior to elements in the training data. The study proposes further training and averaging to measure the sensitivity of a given model to training instances, serving as a gold standard for TDA in the “final-model-only” setting. The authors unify existing gradient-based methods for TDA by showing they approximate this gold standard in different ways. Empirical investigation is conducted on tabular, image, and text datasets and models, revealing that first-order methods can be high-quality but decay with further training, while influence function methods are more stable but lower in quality. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine trying to figure out why a machine learning model makes certain predictions. This paper is about finding the right answers by understanding how the data used to train the model affects its behavior. The researchers propose a new way to measure this effect, which they call “training data attribution”. They test this method and compare it to other ways of doing this task, finding that some methods are better than others in certain situations. |
Keywords
» Artificial intelligence » Machine learning
