Summary of Towards Cross-modal Backward-compatible Representation Learning For Vision-language Models, by Young Kyun Jang et al.
Towards Cross-modal Backward-compatible Representation Learning for Vision-Language Models
by Young Kyun Jang, Ser-nam Lim
First submitted to arxiv on: 23 May 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper addresses a long-standing challenge in modern retrieval systems, where upgrading to newer models necessitates costly processes like backfilling. This problem arises from incompatible embeddings between old and new models. The authors propose Cross-modal Backward-compatible Training (XBT), extending the concept of vision-only BT to the field of cross-modal retrieval. Specifically, they focus on achieving backward-compatibility between Vision-Language Pretraining (VLP) models like CLIP for the cross-modal retrieval task. To tackle XBT challenges, they introduce an efficient projection module that maps new model embeddings to old model embeddings, utilizing text data for pretraining. This approach reduces image-text pairs required for XBT learning and avoids using the old model during training. The authors also leverage parameter-efficient training strategies to preserve off-the-shelf knowledge while improving efficiency. Experimental results on cross-modal retrieval datasets demonstrate the effectiveness of XBT in enabling backfill-free upgrades when new VLP models emerge. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper solves a big problem with modern search systems. When we want to use newer, better models, it’s hard because our old and new models don’t “speak” the same language. The authors find a way to make sure these models can work together seamlessly. They call this idea Cross-modal Backward-compatible Training (XBT). It helps us upgrade our search systems without having to re-do all the work we’ve already done. This means we can use new, better models without losing what we learned from older ones. The authors tested their idea on some datasets and showed that it works well. |
Keywords
» Artificial intelligence » Parameter efficient » Pretraining
