Summary of Clip with Generative Latent Replay: a Strong Baseline For Incremental Learning, by Emanuele Frascaroli et al.
CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning
by Emanuele Frascaroli, Aniello Panariello, Pietro Buzzega, Lorenzo Bonicelli, Angelo Porrello, Simone Calderara
First submitted to arxiv on: 22 Jul 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach to continual learning, called Continual Generative training for Incremental prompt-Learning (CGIPL), is proposed to adapt Vision-Language Models (VLMs) such as CLIP without compromising their original zero-shot capabilities. The method employs Variational Autoencoders (VAEs) to learn class-conditioned distributions within the embedding space of the visual encoder, generating new synthetic visual embeddings and training corresponding textual prompts during subsequent tasks. Through extensive experiments on different domains, CGIPL shows improved zero-shot capabilities and adapts to new tasks while mitigating forgetting. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine you have a really smart language model that can understand pictures! You want to teach it new things without losing what it already knows. The problem is that the model gets very good at understanding what it was trained on, but then forgets how to do the original things it knew. This paper proposes a solution called Continual Generative training for Incremental prompt-Learning (CGIPL). It’s like creating new pictures based on what the model has already learned, so it can keep getting better and better. |
Keywords
* Artificial intelligence * Continual learning * Embedding space * Encoder * Language model * Prompt * Zero shot
