Summary of Tinyllava: a Framework Of Small-scale Large Multimodal Models, by Baichuan Zhou et al.
TinyLLaVA: A Framework of Small-scale Large Multimodal Models
by Baichuan Zhou, Ying Hu, Xi Weng, Junlong Jia, Jie Luo, Xien Liu, Ji Wu, Lei Huang
First submitted to arxiv on: 22 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The TinyLLaVA framework is a unified approach to designing and analyzing small-scale Large Multimodal Models (LMMs). The framework allows researchers to study the effects of different vision encoders, connection modules, language models, training data, and training recipes on LMM performance. Experimental results show that better quality data combined with optimized training recipes can lead to comparable performances from smaller LMMs as larger ones. Under this framework, a family of small-scale LMMs was trained, including the top-performing TinyLLaVA-3.1B model, which outperforms existing 7B models such as LLaVA-1.5 and Qwen-VL. The findings can serve as baselines for future research in data scaling, training setups, and model selection. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary We’re going to build a special kind of computer program called TinyLLaVA. This program helps us design and test smaller versions of large computer models that can understand different types of information like pictures and words. We tested our program with many different combinations of parts and found that using high-quality data and the right way to train the model can make the smaller version work just as well as a bigger one. We even trained a special tiny model called TinyLLaVA-3.1B that does better than some larger models we’ve seen before. We hope this helps other researchers figure out how to make their own computer programs better. |
