Summary of Learning Semantic Proxies From Visual Prompts For Parameter-efficient Fine-tuning in Deep Metric Learning, by Li Ren et al.
Learning Semantic Proxies from Visual Prompts for Parameter-Efficient Fine-Tuning in Deep Metric Learning
by Li Ren, Chen Chen, Liqiang Wang, Kien Hua
First submitted to arxiv on: 4 Feb 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Deep learning has long been a key objective in the machine learning community, with existing solutions focusing on fine-tuning pre-trained models on conventional image datasets. However, recent success of large-scale dataset training makes it challenging to adapt these models to local data domains while retaining previously gained knowledge. To address this challenge, we propose a novel framework for fine-tuning pre-trained Vision Transformers (ViT) using Visual Prompts (VPT). Our approach incorporates semantic information from input images and ViT to optimize visual prompts for each class. We demonstrate that our method outperforms representative capabilities, achieving improved metric learning performance. Evaluating popular DML benchmarks, we show that our framework is effective and efficient, achieving comparable or even better performance than state-of-the-art full fine-tuning works while tuning only a small percentage of total parameters. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Deep learning has made big progress in recognizing pictures! But it’s hard to use these pre-trained models with new data because they were trained on huge datasets. We found a way to make them work better by adding special prompts for each picture class. This helps the model learn more about what makes one picture different from another. Our new method is good at learning how similar or different two pictures are, and it’s faster than other methods that need to adjust all the model parameters. |
Keywords
* Artificial intelligence * Deep learning * Fine tuning * Machine learning * Vit
