Summary of Dam: Dynamic Adapter Merging For Continual Video Qa Learning, by Feng Cheng et al.
DAM: Dynamic Adapter Merging for Continual Video QA Learning
by Feng Cheng, Ziyang Wang, Yi-Lin Sung, Yan-Bo Lin, Mohit Bansal, Gedas Bertasius
First submitted to arxiv on: 13 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Our proposed method, DAM (Dynamic Adapter Merging), tackles the challenge of continual video question-answering (VidQA) learning by efficiently adapting to new datasets while preventing catastrophic forgetting. The approach involves freezing a large pre-trained video-language backbone and training dataset-specific adapters sequentially. At inference time, a non-parametric router function computes adapter relevance for each input instance, which is then aggregated using dynamic adapter merging to produce the final VidQA prediction. Our experiments demonstrate that DAM outperforms prior state-of-the-art continual learning approaches by 9.1% on six VidQA datasets spanning various domains. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Our paper introduces a new way to learn from videos and questions without forgetting what we already know. We show how to adapt our model to new video-question pairs while keeping the most important parts of our knowledge. This is useful for applications where there are many different types of data, such as educational videos or sports highlights. Our method works well on various datasets and outperforms previous approaches. |
Keywords
* Artificial intelligence * Continual learning * Inference * Question answering
