Summary of Effectiveness Assessment Of Recent Large Vision-language Models, by Yao Jiang et al.
Effectiveness Assessment of Recent Large Vision-Language Models
by Yao Jiang, Xinyu Yan, Ge-Peng Ji, Keren Fu, Meijun Sun, Huan Xiong, Deng-Ping Fan, Fahad Shahbaz Khan
First submitted to arxiv on: 7 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The advent of large vision-language models (LVLMs) marks a significant milestone in the pursuit of artificial general intelligence. However, their efficacy in both specialized and general tasks remains unclear. This paper investigates the competency of popular LVLMs in specialized and general tasks, aiming to provide a comprehensive understanding of these novel models. The study evaluates three recent open-source LVLMs – MiniGPT-v2, LLaVA-1.5, and Shikra – on various visual recognition and localization tasks in natural, healthcare, and industrial scenarios. Additionally, the paper explores the multi-modal understanding capabilities of these LVLMs in general tasks like object counting, absurd question answering, affordance reasoning, attribute recognition, and spatial relation reasoning. The results reveal that these LVLMs demonstrate limited proficiency in both specialized and general tasks, highlighting potential factors such as limited cognition, object hallucination, text-to-image interference, and decreased robustness in complex problems. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large vision-language models are super smart computers that can understand images and words. But how good are they at doing specific jobs? And how good are they at understanding things in general? This study looked at three special kinds of these computer models to see how well they did on various tasks, like recognizing objects or answering questions. The results showed that these computers weren’t very good at most tasks, and it’s because they have some limitations, like not being able to understand certain things or getting confused. |
Keywords
* Artificial intelligence * Hallucination * Multi modal * Question answering
