Summary of Data Selection For Transfer Unlearning, by Nazanin Mohammadi Sepahvand et al.
Data Selection for Transfer Unlearning
by Nazanin Mohammadi Sepahvand, Vincent Dumoulin, Eleni Triantafillou, Gintare Karolina Dziugaite
First submitted to arxiv on: 16 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary As machine learning models grow in complexity and data requirements, concerns arise over the use of training data, particularly when agreements change over time. This issue has driven attention to machine unlearning: removing the influence of a subset of training data from a trained model. Our work advocates for a relaxed definition of unlearning that targets scenarios where data owners withdraw permission for training purposes. We focus on transfer unlearning, where a pretrained model is transferred to a target dataset containing some non-static data that may need to be unlearned in the future. We propose a new method using an auxiliary “static” dataset and finetuning selected examples instead of the target data; addressing all unlearning requests ahead of time. Our approach outperforms the gold standard exact unlearning on several datasets, especially for small static sets, sometimes approaching an upper bound for test accuracy. Factors influencing the accuracy boost obtained by data selection are also analyzed. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Machine learning models are getting bigger and need more data. This is causing problems because data owners might change their minds about letting us use their data. This issue has led to a new area of research called machine unlearning, which is like deleting old memories from a trained model. Our work focuses on a specific problem called transfer unlearning, where we take a pre-trained model and move it to a new dataset that contains some data that might need to be deleted in the future. We came up with a new way to do this by using an auxiliary dataset and selecting certain examples to fine-tune instead of the whole target dataset. Our method works well on many datasets, especially when the auxiliary dataset is small. |
Keywords
» Artificial intelligence » Attention » Machine learning
