Summary of Bivlc: Extending Vision-language Compositionality Evaluation with Text-to-image Retrieval, by Imanol Miranda et al.
BiVLC: Extending Vision-Language Compositionality Evaluation with Text-to-Image Retrieval
by Imanol Miranda, Ander Salaberria, Eneko Agirre, Gorka Azkune
First submitted to arxiv on: 14 Jun 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Computation and Language (cs.CL); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a new benchmark for Vision-Language Compositionality (VLC) called BiVLC, which adds a synthetic hard negative image generated from the synthetic text. This results in two image-to-text retrieval examples and two text-to-image retrieval examples. The novelty of BiVLC is that it challenges current multimodal models to perform well in both directions, unlike existing benchmarks like SugarCrepe, which only focus on image-to-text retrieval. The authors show that a contrastive model trained using synthetic images and texts significantly improves over the base model in BiVLC for both retrieval directions. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The new BiVLC benchmark is designed to test multimodal models’ ability to perform well in both text-to-image and image-to-text retrieval tasks. This is different from existing benchmarks like SugarCrepe, which only focus on image-to-text retrieval. The authors show that current multimodal models struggle with the new BiVLC benchmark, especially when it comes to text-to-image retrieval. They also propose a contrastive model trained using synthetic images and texts, which outperforms the base model in both directions. |
