Summary of Enhancing Vision-language Model Pre-training with Image-text Pair Pruning Based on Word Frequency, by Mingliang Liang et al.
Enhancing Vision-Language Model Pre-training with Image-text Pair Pruning Based on Word Frequency
by Mingliang Liang, Martha Larson
First submitted to arxiv on: 9 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary We propose Word-Frequency-based Image-Text Pair Pruning (WFPP), a novel method that improves the efficiency of Vision-and-Language Models (VLMs). Unlike MetaCLIP, WFPP prunes text-image pairs based on word frequencies across the entire training dataset. This reduces the dominance of frequent words, resulting in a better-balanced word-frequency distribution. Our experiments demonstrate that applying WFPP when training a CLIP model improves performance on various downstream tasks, including speeding up pre-training using fewer samples. We also analyze the training data before and after pruning to visualize the changes in word frequencies. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine if you could make computers learn faster by getting rid of unnecessary information. That’s what this team did! They created a new way to help machines understand pictures and words better, called Word-Frequency-based Image-Text Pair Pruning (WFPP). It works by removing text-image pairs that have too many common words, making it easier for the machine to learn. This helps computers perform tasks like recognizing objects in images or understanding what people are saying. The team showed that this method makes machines better at doing these tasks and also speeds up the learning process. |
Keywords
» Artificial intelligence » Pruning
