Summary of The Solution For Language-enhanced Image New Category Discovery, by Haonan Xu et al.
The Solution for Language-Enhanced Image New Category Discovery
by Haonan Xu, Dian Chao, Xiangyu Wu, Zhonghua Wan, Yang Yang
First submitted to arxiv on: 6 Jul 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach to improving zero-shot multi-label image recognition is proposed in this paper, which combines techniques from natural language processing and computer vision. The authors introduce Pseudo Visual Prompts, initialized for each object category and pre-trained on sentence data generated by large language models. These prompts are then used to enhance the visual representation capacity of textual labels through contrastive learning. Additionally, a dual-adapter module is introduced that leverages knowledge from both the original CLIP model and new learning knowledge derived from downstream datasets. The proposed method surpasses state-of-the-art performance on clean annotated text data as well as pseudo text data generated by large language models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper proposes a new way to help computers recognize objects in images without being specifically trained for that task. It does this by using words and sentences to teach the computer what different things look like. The authors create special “prompts” for each type of object, which are then used to make the text more able to represent visual information. This allows the text to be used to help with image recognition tasks, and it performs better than other methods that have been tried. |
Keywords
* Artificial intelligence * Natural language processing * Zero shot
