Summary of Attribute-to-delete: Machine Unlearning Via Datamodel Matching, by Kristian Georgiev et al.
Attribute-to-Delete: Machine Unlearning via Datamodel Matching
by Kristian Georgiev, Roy Rinberg, Sung Min Park, Shivam Garg, Andrew Ilyas, Aleksander Madry, Seth Neel
First submitted to arxiv on: 30 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Machine learning educators can summarize this research paper by saying: This study proposes a new approach called Datamodel Matching (DMM) to efficiently remove the effect of a small training data set on pre-trained machine learning models. The goal is to produce a model that outputs are statistically indistinguishable from those of a model re-trained on all but the forget set. DMM uses data attribution to predict the output if the model were re-trained, then fine-tunes the pre-trained model to match these predicted outputs. In convex settings, this approach outperforms existing iterative unlearning algorithms, and in non-convex settings, it achieves strong unlearning performance relative to existing algorithms. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Machine learning is like trying to remember something from a long time ago. Sometimes we need to “forget” things that are no longer important. Researchers have been working on ways to do this efficiently with machine learning models. This study proposes a new way called Datamodel Matching (DMM) that works well even when the problem is hard. It uses an idea called data attribution, which helps predict what would happen if we re-trained the model without some of the old information. Then it fine-tunes the original model to match these predictions. This approach does better than other methods in certain situations and has potential for future improvements. |
Keywords
* Artificial intelligence * Machine learning
