Summary of Sphinx-x: Scaling Data and Parameters For a Family Of Multi-modal Large Language Models, by Dongyang Liu et al.
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models
by Dongyang Liu, Renrui Zhang, Longtian Qiu, Siyuan Huang, Weifeng Lin, Shitian Zhao, Shijie Geng, Ziyi Lin, Peng Jin, Kaipeng Zhang, Wenqi Shao, Chao Xu, Conghui He, Junjun He, Hao Shao, Pan Lu, Hongsheng Li, Yu Qiao, Peng Gao
First submitted to arxiv on: 8 Feb 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes an extensive series of Multimodality Large Language Models (MLLMs) called SPHINX-X, which improves upon the original SPHINX framework by simplifying the architecture and training process. The authors create a comprehensive multimodal dataset covering various tasks, including language, vision, and vision-language, as well as curate additional datasets for OCR-intensive and set-of-mark tasks. By training MLLMs on different base models, they obtain a spectrum of models varying in parameter size and multilingual capabilities. Benchmarking reveals a strong correlation between performance and data/parameter scales. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper creates a new type of language model called SPHINX-X. They make changes to the original SPHINX model to make it better. They also collect lots of different types of data, including pictures and text. This helps their models learn more things. They trained many different models on this data and found that bigger models with more information did better. |
Keywords
* Artificial intelligence * Language model
