Summary of Blink: Multimodal Large Language Models Can See but Not Perceive, by Xingyu Fu et al.
BLINK: Multimodal Large Language Models Can See but Not Perceive
by Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma, Ranjay Krishna
First submitted to arxiv on: 18 Apr 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The researchers introduce Blink, a benchmark that tests core visual perception abilities of multimodal language models (LLMs). The tasks are designed to be solvable by humans within a short time frame, but challenging for current LLMs. The authors reformatted 14 classic computer vision tasks into 3,807 multiple-choice questions, paired with images and visual prompts. While humans achieve high accuracy on average, existing multimodal LLMs struggle, even the best-performing ones achieving only 13-15% higher than random guessing. This highlights potential pathways for future improvements. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Blink is a new test for language models that looks at how well they can understand what they see. The tasks are hard for machines to do but easy for people to solve quickly. The researchers took classic computer vision problems and turned them into questions with pictures, making it harder for computers to get the right answers. People got most of the questions correct, but even the best language models only did a little better than guessing. This shows that these models still need help to understand what they see. |
