Summary of Unified Text-to-image Generation and Retrieval, by Leigang Qu et al.
Unified Text-to-Image Generation and Retrieval
by Leigang Qu, Haochuan Li, Tan Wang, Wenjie Wang, Yongqi Li, Liqiang Nie, Tat-Seng Chua
First submitted to arxiv on: 9 Jun 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multimedia (cs.MM)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed framework unites text-to-image generation and retrieval using Multimodal Large Language Models (MLLMs), enhancing the creation of diverse visual content. By leveraging MLLMs’ discriminative abilities, the authors introduce a generative retrieval method that performs retrieval without training. The approach combines generation and retrieval in an autoregressive manner, featuring an autonomous decision module to choose the best-matched image between generated and retrieved ones. A benchmark, TIGeR-Bench, is constructed for evaluating unified text-to-image generation and retrieval. Experimental results on TIGeR-Bench, Flickr30K, and MS-COCO demonstrate the proposed method’s superiority. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper explores how to efficiently create new images from texts. Right now, we can retrieve existing images from a database using text queries, but these databases are limited in their creativity. A different approach is generating new images from texts, which has made some progress but still struggles with creating complex, informative images. This research proposes a way to combine both methods and create more diverse and accurate image responses to text inputs. The authors test this method on various datasets and show that it outperforms existing approaches. |
Keywords
» Artificial intelligence » Autoregressive » Image generation
