Summary of Vcd: Knowledge Base Guided Visual Commonsense Discovery in Images, by Xiangqing Shen et al.
VCD: Knowledge Base Guided Visual Commonsense Discovery in Images
by Xiangqing Shen, Yurun Song, Siwei Wu, Rui Xia
First submitted to arxiv on: 27 Feb 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents a novel approach to discovering visual commonsense, which encompasses knowledge about object properties, relationships, and behaviors in images. The existing definitions of visual commonsense are coarse-grained and incomplete, so the authors introduce a new task called Visual Commonsense Discovery (VCD) to extract fine-grained commonsense from different objects within an image. They construct a dataset featuring over 100,000 images and 14 million object-commonsense pairs using Visual Genome and ConceptNet. The authors also propose a generative model that integrates a vision-language model with instruction tuning to tackle VCD. Experimental results demonstrate the model’s proficiency in VCD, particularly outperforming GPT-4V in implicit commonsense discovery. The paper further highlights the value of VCD by applying it to two downstream tasks: visual commonsense evaluation and visual question answering. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about finding common sense in pictures. Common sense is what you know from your everyday life, like that cats are usually small or that cars can drive on roads. The authors want to find this kind of knowledge in images, so they created a new task called Visual Commonsense Discovery (VCD). They also made a dataset with many images and common sense facts, and proposed a way to use computers to do VCD better than before. |
Keywords
» Artificial intelligence » Generative model » Gpt » Instruction tuning » Language model » Question answering
