Summary of Hourvideo: 1-hour Video-language Understanding, by Keshigeyan Chandrasegaran et al.
HourVideo: 1-Hour Video-Language Understanding
by Keshigeyan Chandrasegaran, Agrim Gupta, Lea M. Hadzic, Taran Kota, Jimming He, Cristóbal Eyzaguirre, Zane Durante, Manling Li, Jiajun Wu, Li Fei-Fei
First submitted to arxiv on: 7 Nov 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The authors introduce HourVideo, a large-scale dataset for video-language understanding, focusing on tasks that require comprehension of long-duration videos (20-120 minutes). The benchmark comprises summarization, perception, visual reasoning, and navigation tasks, with 500 manually curated egocentric videos featuring high-quality, five-way multiple-choice questions. While multimodal models show marginal improvements over random chance, human experts significantly outperform the state-of-the-art Gemini Pro 1.5 model (85.0% vs. 37.3%). The dataset and evaluation toolkit are available online. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a big challenge for computers to understand hour-long videos. They made a special test called HourVideo that has lots of tasks like summarizing what’s happening in the video, recognizing objects or people, and even understanding complex things like cause-and-effect relationships. The goal is to see how well different computer models can do these tasks compared to humans. So far, computers are not very good at it! |
Keywords
» Artificial intelligence » Gemini » Language understanding » Summarization
