Summary of Reconstructing Training Data From Real World Models Trained with Transfer Learning, by Yakir Oz et al.
Reconstructing Training Data From Real World Models Trained with Transfer Learning
by Yakir Oz, Gilad Yehudai, Gal Vardi, Itai Antebi, Michal Irani, Niv Haim
First submitted to arxiv on: 22 Jul 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach is proposed to reconstruct training data from trained classifiers, enabling data reconstruction in realistic settings for models trained on high-resolution images. The method adapts the reconstruction scheme of arXiv:2206.07758 to real-world scenarios, targeting models trained via transfer learning over image embeddings of large pre-trained models like DINO-ViT and CLIP. Data reconstruction is employed in the embedding space, showcasing its applicability beyond visual data. A novel clustering-based method is introduced to identify good reconstructions from thousands of candidates, improving on previous works that relied on knowledge of the training set to identify good reconstructed images. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine a way to see what your trained AI model learned from, even if it was trained on super high-quality pictures! This new approach makes it possible to do just that. It uses clever tricks from another study (arXiv:2206.07758) and applies them to real-world situations where models are trained using big pre-trained models like DINO-ViT and CLIP. The cool thing is that this method works not just for pictures, but for any kind of data that can be embedded into a special space. It also introduces a new way to pick the best reconstructed versions from many possible choices. |
Keywords
» Artificial intelligence » Clustering » Embedding space » Transfer learning » Vit
