Summary of Sco-vist: Social Interaction Commonsense Knowledge-based Visual Storytelling, by Eileen Wang et al.
SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytelling
by Eileen Wang, Soyeon Caren Han, Josiah Poon
First submitted to arxiv on: 1 Feb 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces SCO-VIST, a novel framework for generating coherent and engaging visual stories from image sequences. Unlike previous models that mainly focus on applying factual information, SCO-VIST incorporates human action motivation and social interaction commonsense knowledge to create bridges between plot points. The weighted story graph produced by SCO-VIST yields superior results in terms of visual grounding, coherence, diversity, and humanness across multiple metrics. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine you want a computer to tell a story based on a series of images. This is called “visual storytelling.” Most computers are not very good at this because they just repeat what’s in the pictures without making sense of them. The researchers created a new way for computers to make stories by thinking about why people do things and how those actions relate to each other. Their method, SCO-VIST, produces much better stories than existing methods. It can create stories that are visually grounded (accurate), coherent (make sense), diverse (interesting), and even “human-like” in their storytelling style. |
Keywords
» Artificial intelligence » Grounding
