Summary of Visual Delta Generator with Large Multi-modal Models For Semi-supervised Composed Image Retrieval, by Young Kyun Jang et al.
Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrieval
by Young Kyun Jang, Donghyun Kim, Zihang Meng, Dat Huynh, Ser-Nam Lim
First submitted to arxiv on: 23 Apr 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a new semi-supervised Composed Image Retrieval (CIR) approach that leverages large language model-based Visual Delta Generator (VDG) to generate text describing the visual difference between reference and target images. The proposed method, which uses pseudo triplets generated by VDG, significantly outperforms existing supervised learning approaches on CIR benchmarks. This is achieved by searching for reference and target images in auxiliary data and using them to train the VDG. The authors demonstrate that their approach can improve the performance of CIR models without requiring a large number of labeled triplets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a new way to find pictures that are similar to each other based on a description. Usually, we need lots of labeled pictures and descriptions to make this work well, but this method uses a different approach. It takes in some extra information about the pictures and then uses it to create fake training data that makes the system better at finding the right pictures. This method is really good at doing this task and beats other ways of doing it. |
Keywords
» Artificial intelligence » Large language model » Semi supervised » Supervised
