Summary of Feature Protection For Out-of-distribution Generalization, by Lu Tan et al.
Feature Protection For Out-of-distribution Generalization
by Lu Tan, Huei Zhou, Yinxiang Huang, Zeming Zheng, Yujiu Yang
First submitted to arxiv on: 25 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the performance of fine-tuning large pre-trained models for real-world machine learning applications. The authors highlight the challenge of designing fine-tuning methods that are robust to out-of-distribution (OOD) data, which is under-represented by the training dataset. They show that standard fine-tuning methods lead to overfitting and deteriorated OOD performance. To address this issue, the authors propose feature protection methods that protect pre-trained features, resulting in a more robust model for OOD generalization. The study validates these methods through extensive experiments with CLIP on ImageNet and DomainNet. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how to make machine learning models work better when they’re used in new situations or with data that’s not exactly like what they were trained on. Right now, most people fine-tune big pre-trained models by adjusting them a little bit for their specific task. But this can cause the model to become too specialized and not work well when it encounters new data. The authors of this paper propose ways to “protect” the original features of these models so that they’re more robust and don’t overfit to the training data. |
Keywords
» Artificial intelligence » Fine tuning » Generalization » Machine learning » Overfitting
