Summary of Llava-onevision: Easy Visual Task Transfer, by Bo Li et al.
LLaVA-OneVision: Easy Visual Task Transfer
by Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, Chunyuan Li
First submitted to arxiv on: 6 Aug 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces LLaVA-OneVision, a suite of open large multimodal models (LMMs) that excel in three computer vision scenarios: single-image, multi-image, and video. These models consolidate insights from data, visual representations, and previous blog posts. The experimental results show that LLaVA-OneVision sets new performance boundaries for open LMMs across these scenarios. Moreover, the design enables strong transfer learning between different modalities/scenarios, leading to novel emerging capabilities. Specifically, the model demonstrates robust video understanding and cross-scenario capabilities through task transfer from images to videos. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper presents a family of large multimodal models called LLaVA-OneVision. These models can do many things at once, like recognizing objects in single pictures, groups of pictures, and even videos. This is the first model that can do all these things so well. The researchers found that their design makes it easy for the model to learn from one type of image and apply what it learned to another. For example, they showed that the model can recognize objects in images and then use this skill to understand videos. |
Keywords
* Artificial intelligence * Transfer learning
