Summary of Rec-gpt4v: Multimodal Recommendation with Large Vision-language Models, by Yuqing Liu et al.
Rec-GPT4V: Multimodal Recommendation with Large Vision-Language Models
by Yuqing Liu, Yu Wang, Lichao Sun, Philip S. Yu
First submitted to arxiv on: 13 Feb 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel reasoning scheme called Rec-GPT4V: Visual-Summary Thought (VST) for multimodal recommendations using large vision-language models (LVLMs). LVLMs are proficient in understanding static images and textual dynamics, but their application in this field is limited due to the lack of user preference knowledge and difficulties in addressing multiple image dynamics. The VST scheme leverages LVLMs by utilizing user history as in-context user preferences, generating item image summaries, and querying user preferences over candidate items using image comprehension in natural language space combined with item titles. Experimental results across four datasets with three LVLMs (GPT4-V, LLaVa-7b, and LLaVa-13b) demonstrate the efficacy of VST. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about how to use big models that can understand pictures and text to recommend things like movies or music based on what someone likes. The problem with these models is they don’t know what you like because they were trained on lots of random data, not just your favorite shows. They also struggle when there are multiple images or noisy information. To fix this, the authors created a new way called VST that uses user history to figure out what someone likes and then uses the model to generate summaries of pictures and text to help make recommendations. |
