Summary of Improving Zero-shot Generalization Of Learned Prompts Via Unsupervised Knowledge Distillation, by Marco Mistretta et al.
Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation
by Marco Mistretta, Alberto Baldrati, Marco Bertini, Andrew D. Bagdanov
First submitted to arxiv on: 3 Jul 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel approach to prompt learning, called Knowledge Distillation Prompt Learning (KDPL), which eliminates the need for labeled examples during adaptation. By distilling knowledge from more powerful models without annotations, KDPL can improve the generalization of learned prompts for various tasks such as zero-shot domain generalization, cross-dataset generalization, and base-to-novel class generalization. The authors demonstrate the effectiveness of KDPL on over ten standard benchmark datasets, showcasing its potential to transfer knowledge even in the absence of training class names. This technique can be integrated into existing prompt learning methods and has far-reaching implications for adapting Vision-Language Models (VLMs) to unseen tasks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper finds a new way to help computers learn new things without needing lots of examples to practice with. They call this method “Knowledge Distillation Prompt Learning” or KDPL. KDPL is special because it can take the knowledge from more powerful computers and teach it to less powerful ones, without needing any labeled training data. This means that computers could learn to do new tasks all on their own, without human help. The researchers tested KDPL with many different datasets and found that it worked really well. |
Keywords
» Artificial intelligence » Domain generalization » Generalization » Knowledge distillation » Prompt » Zero shot
