Summary of Movie2story: a Framework For Understanding Videos and Telling Stories in the Form Of Novel Text, by Kangning Li et al.
Movie2Story: A framework for understanding videos and telling stories in the form of novel text
by Kangning Li, Zheyang Jia, Anyu Ying
First submitted to arxiv on: 19 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a new benchmark for evaluating text generation capabilities in complex scenarios involving long videos and rich auxiliary information. The Multi-modal Story Generation Benchmark (MSBench) aims to assess comprehension abilities of large-scale models by generating novel evaluation datasets through automated processes and refining auxiliary data through systematic filtering. State-of-the-art models are used to ensure fairness and accuracy of ground-truth datasets. Current MLLMs perform suboptimally under the proposed evaluation metrics, highlighting significant gaps in their capabilities. To address these challenges, a novel model architecture and methodology are proposed, demonstrating improvements on the benchmark. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper creates a new way to test big language models that can understand complex videos and rich information. The Multi-modal Story Generation Benchmark (MSBench) helps evaluate how well these models do in generating text based on what they see and hear. The authors use existing datasets and automated processes to make new evaluation datasets, which saves time and ensures accuracy. Current state-of-the-art models don’t perform very well under the new test metrics, showing there’s still much work to be done. To fix this, the authors propose a new model architecture and approach that can do better on the benchmark. |
Keywords
» Artificial intelligence » Multi modal » Text generation
