Summary of Spurious Feature Eraser: Stabilizing Test-time Adaptation For Vision-language Foundation Model, by Huan Ma et al.
Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model
by Huan Ma, Yan Zhu, Changqing Zhang, Peilin Zhao, Baoyuan Wu, Long-Kai Huang, Qinghua Hu, Bingzhe Wu
First submitted to arxiv on: 1 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The abstract presents a study that investigates the limitations of vision-language foundation models, specifically CLIP, in downstream tasks like fine-grained image classification. Despite their scalability and success on extensive paired data, these models exhibit “decision shortcuts” that hinder their generalization capabilities. The authors identify two types of features: desired invariant causal features and undesired decision shortcuts. They propose a method called Spurious Feature Eraser (SEraser) to alleviate the decision shortcuts by erasing spurious features during inference. This approach involves test-time prompt tuning to optimize learnable prompts, compelling the model to utilize invariant features while disregarding decision shortcuts. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary In simple terms, this study explores why vision-language models are not as good at specific tasks like image classification as they seem. The researchers found that these models have both helpful and unhelpful features. They developed a method called Spurious Feature Eraser (SEraser) to remove the unhelpful features and help the model focus on what’s important. |
Keywords
* Artificial intelligence * Generalization * Image classification * Inference * Prompt
