Summary of Model Reprogramming Outperforms Fine-tuning on Out-of-distribution Data in Text-image Encoders, by Andrew Geng et al.
Model Reprogramming Outperforms Fine-tuning on Out-of-distribution Data in Text-Image Encoders
by Andrew Geng, Pin-Yu Chen
First submitted to arxiv on: 16 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores the effects of fine-tuning techniques on pre-trained models when applied to downstream tasks. The authors highlight the importance of assessing both in-distribution accuracy and out-of-distribution (OOD) generalization and detection capabilities. They introduce a new model reprogramming approach called Reprogrammer, which aims to improve holistic performance across ID, OOD generalization, and OOD detection tasks. Experimental results show that Reprogrammer is less intrusive and yields better downstream models. By appending an additional representation residual connection to Reprogrammer, the authors demonstrate further preservation of pre-training representations, leading to more robust models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper investigates how fine-tuning affects pre-trained models when used in different tasks. It’s crucial to consider not just how well a model does on its original task but also how it performs on new tasks that are similar or very different. The authors introduce a new way to adjust the model, called Reprogrammer, which helps the model do better overall. Tests show that this approach is less disruptive and produces better results. |
Keywords
* Artificial intelligence * Fine tuning * Generalization
