Summary of Teasergen: Generating Teasers For Long Documentaries, by Weihan Xu et al.
TeaserGen: Generating Teasers for Long Documentaries
by Weihan Xu, Paul Pu Liang, Haven Kim, Julian McAuley, Taylor Berg-Kirkpatrick, Hao-Wen Dong
First submitted to arxiv on: 8 Oct 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents DocumentaryNet, a dataset of 1,269 documentaries paired with their teasers, which enables long-range multimodal modeling for teaser generation. The authors propose TeaserGen, a two-stage system that generates narration from the documentary’s transcribed narration using a large language model and then selects relevant visual content through language-vision models. The paper evaluates two approaches for narration-video matching: pretraining-based models using contrastive language-vision models and deep sequential models learning the mapping between narrations and visuals. Results show that the pretraining-based approach is more effective than directly trained autoregressive models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a big dataset of documentaries with their teasers, which helps computers learn to make good teasers for long videos. The authors also develop a system called TeaserGen that can generate narration for a teaser and find the right video clips to go with it. They test two ways to match the audio with the video: using pre-trained models or training a new model from scratch. They find that the pre-trained approach works better. |
Keywords
» Artificial intelligence » Autoregressive » Large language model » Pretraining
