Summary of Mopd: Mixture-of-prompts Distillation For Vision-language Models, by Yang Chen et al.
MoPD: Mixture-of-Prompts Distillation for Vision-Language Models
by Yang Chen, Shuai Fu, Yu Zhang
First submitted to arxiv on: 26 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Computation and Language (cs.CL); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel approach called Mixture-of-Prompts Distillation (MoPD) to improve the generalization ability of soft prompt learning methods for vision-language models (VLMs). Existing methods tend to overfit seen classes and perform poorly on unseen classes, which is attributed to the bias in training data. MoPD addresses this issue by transferring knowledge from hand-crafted teacher prompts to learnable student prompts, enhancing performance on unseen classes. The method utilizes a gating network that selects hard prompts for prompt distillation. Experimental results show that MoPD outperforms state-of-the-art baselines, particularly on unseen classes. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper wants to help machines learn better by creating new ways to teach them. Right now, machines are good at doing what they were trained to do, but not so good at doing something new. The problem is that the training data has a bias towards what the machine was originally taught, making it hard for the machine to learn something new. To solve this issue, the researchers propose a new method called Mixture-of-Prompts Distillation (MoPD). MoPD takes the knowledge from the original teacher and applies it to the student, allowing the machine to learn better even when faced with something new. |
Keywords
» Artificial intelligence » Distillation » Generalization » Prompt
