Summary of Cliperase: Efficient Unlearning Of Visual-textual Associations in Clip, by Tianyu Yang et al.
CLIPErase: Efficient Unlearning of Visual-Textual Associations in CLIP
by Tianyu Yang, Lisen Dai, Zheyuan Liu, Xiangqi Wang, Meng Jiang, Yapeng Tian, Xiangliang Zhang
First submitted to arxiv on: 30 Oct 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Machine unlearning (MU) has gained significant attention as a means to remove specific data from trained models without requiring a full retraining process. In this work, we address the unique challenges of unlearning in CLIP, a prominent multimodal model that aligns visual and textual representations. We introduce CLIPErase, a novel approach that disentangles and selectively forgets both visual and textual associations, ensuring that unlearning does not compromise model performance. The approach consists of three key modules: a Forgetting Module that disrupts the associations in the forget set, a Retention Module that preserves performance on the retain set, and a Consistency Module that maintains consistency with the original model. Extensive experiments on the CIFAR-100 and Flickr30K datasets across four CLIP downstream tasks demonstrate that CLIPErase effectively forgets designated associations in zero-shot tasks for multimodal samples, while preserving the model’s performance on the retain set after unlearning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Machine unlearning (MU) is a way to remove specific data from trained models without retraining them. This is important because it can help make sure that our models are not biased or unfair. In this paper, we focus on making MU work for multimodal models like CLIP, which connects images and text. We create a new approach called CLIPErase that helps these models forget certain associations while still being good at what they do. This is useful because it can help us use our models in new ways without worrying about them remembering old things we didn’t want them to remember. |
Keywords
* Artificial intelligence * Attention * Zero shot
