Summary of The Synergy Between Data and Multi-modal Large Language Models: a Survey From Co-development Perspective, by Zhen Qin et al.
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective
by Zhen Qin, Daoyuan Chen, Wenhao Zhang, Liuyi Yao, Yilun Huang, Bolin Ding, Yaliang Li, Shuiguang Deng
First submitted to arxiv on: 11 Jul 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates the development of large language models (LLMs) and multi-modal LLMs (MLLMs), which extend their capabilities beyond text to various domains. MLLMs rely on vast amounts of data, highlighting the importance of data in achieving emergent capabilities. Recent works find that model development and data development are interconnected, with better data leading to improved MLLM performance and MLLMs facilitating data development. The paper reviews existing works related to MLLMs from a data-model co-development perspective. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper explores how large language models (LLMs) can be extended to handle various types of data. It looks at the connection between LLMs and multi-modal LLMs, which can process images, audio, or other forms of data. The authors find that these models rely on huge amounts of information to work well. They also discover that developing better models and better data go hand in hand. |
Keywords
* Artificial intelligence * Multi modal
