Summary of Dense Connector For Mllms, by Huanjin Yao et al.
Dense Connector for MLLMs
by Huanjin Yao, Wenhao Wu, Taojiannan Yang, YuXin Song, Mengxi Zhang, Haocheng Feng, Yifan Sun, Zhiheng Li, Wanli Ouyang, Jingdong Wang
First submitted to arxiv on: 22 May 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Medium Difficulty Summary: A recent surge in Multimodal Large Language Model (MLLM) performance has led to a focus on linguistic aspects, while visual signals are often overlooked. This paper introduces the Dense Connector, a simple and effective vision-language connector that leverages multi-layer visual features, adding minimal computational overhead to existing MLLMs. The authors also propose the Efficient Dense Connector, achieving comparable performance to LLaVA-v1.5 with only 25% of the visual tokens. Zero-shot video understanding capabilities are demonstrated without additional training. Experiments validate the approach’s versatility and scalability across various vision encoders, image resolutions, training dataset scales, and MLLM architectures, achieving state-of-the-art performance on 19 image and video benchmarks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Low Difficulty Summary: Imagine having a superpower that lets computers understand images and videos better. Right now, most computer models are only good at understanding words, not pictures or moving pictures. This paper helps fix that by creating a new tool called the Dense Connector. It makes existing language models (which are already really good at understanding words) even better at understanding images and videos. The best part is it doesn’t require a lot of extra computer power. The authors also show that this tool can be used for video understanding without needing to train it on lots of new data. |
Keywords
» Artificial intelligence » Large language model » Zero shot
