Summary of Punchbench: Benchmarking Mllms in Multimodal Punchline Comprehension, by Kun Ouyang et al.
PunchBench: Benchmarking MLLMs in Multimodal Punchline Comprehension
by Kun Ouyang, Yuanxin Liu, Shicheng Li, Yi Liu, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun
First submitted to arxiv on: 16 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces a novel benchmark for assessing multimodal large language models’ (MLLMs) ability to comprehend punchlines, which are humorous or sarcastic image-caption pairs. The existing benchmarks have limitations, including reliance on text shortcuts, limited question diversity, and narrow domain focus. To address these issues, the authors propose PunchBench, a comprehensive benchmark that includes diverse question formats and captions from various domains. The authors also introduce a Simple-to-Complex Chain-of-Question (SC-CoQ) strategy to improve MLLMs’ punchline comprehension performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper aims to create a better way to test how well computers can understand funny or sarcastic image-text pairs, called punchlines. The current methods for testing this have some big problems, like relying too much on text and not having enough different questions. To fix these issues, the authors created a new benchmark called PunchBench that includes many different types of questions and captions from various domains. They also came up with a strategy to help computers get better at understanding punchlines by gradually building their knowledge. |
