Summary of Amu-tuning: Effective Logit Bias For Clip-based Few-shot Learning, by Yuwei Tang et al.
AMU-Tuning: Effective Logit Bias for CLIP-based Few-shot Learning
by Yuwei Tang, Zhenyi Lin, Qilong Wang, Pengfei Zhu, Qinghua Hu
First submitted to arxiv on: 13 Apr 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Computation and Language (cs.CL); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores the potential of pre-trained vision-language models like CLIP in few-shot learning. Despite efforts to improve their performance, key factors influencing their effectiveness have not been well understood. The authors introduce a unified formulation to analyze CLIP-based methods from the perspective of logit bias, encouraging the development of an effective logit bias for improved performance. They disassemble three components involved in logit bias computation and empirically analyze their effect on few-shot classification performance. Based on this analysis, they propose AMU-Tuning, a novel method to learn effective logit bias for CLIP-based few-shot classification. This involves predicting logit bias using auxiliary features fed into an efficient feature-initialized linear classifier with multi-branch training. Finally, an uncertainty-based fusion is developed to incorporate logit bias into CLIP for few-shot classification. Experimental results on several benchmarks show that AMU-Tuning outperforms its counterparts and achieves state-of-the-art performance without additional components. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper talks about how computers can learn new things quickly by using special models called vision-language models. These models are good at recognizing images, but they also understand words and sentences. The goal is to make them even better at learning new things fast. To do this, the researchers looked at what makes some methods better than others for these models. They found that one important thing is how well the model can adjust its answers based on the words it sees. Then, they came up with a new way to help the model do this better using special features and training techniques. This new method works really well and helps the model learn even faster. |
Keywords
* Artificial intelligence * Classification * Few shot
