Summary of Distribution-level Memory Recall For Continual Learning: Preserving Knowledge and Avoiding Confusion, by Shaoxu Cheng et al.
Distribution-Level Memory Recall for Continual Learning: Preserving Knowledge and Avoiding Confusion
by Shaoxu Cheng, Kanglei Geng, Chiyuan He, Zihuan Qiu, Linfeng Xu, Heqian Qiu, Lanxiao Wang, Qingbo Wu, Fanman Meng, Hongliang Li
First submitted to arxiv on: 4 Aug 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Continual Learning (CL) aims to enable Deep Neural Networks (DNNs) to learn new data without forgetting previously learned knowledge. The key to achieving this goal is to avoid confusion at the feature level, i.e., avoiding confusion within old tasks and between new and old tasks. To address this issue, we propose the Distribution-Level Memory Recall (DMR) method, which uses a Gaussian mixture model to precisely fit the feature distribution of old knowledge at the distribution level and generate pseudo features in the next stage. Additionally, resistance to confusion at the distribution level is crucial for multimodal learning, as the problem of multimodal imbalance results in significant differences in feature responses between different modalities, exacerbating confusion within old tasks. Therefore, we mitigate the multi-modal imbalance problem by using the Inter-modal Guidance and Intra-modal Mining (IGIM) method to guide weaker modalities with prior information from dominant modalities and further explore useful information within modalities. Furthermore, we propose the Confusion Index to quantitatively describe a model’s ability to distinguish between new and old tasks, and use the Incremental Mixup Feature Enhancement (IMFE) method to enhance pseudo features with new sample features, alleviating classification confusion between new and old knowledge. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about how machines can learn from new data without forgetting what they already know. The key is to avoid getting confused when learning new things. To do this, the authors propose a method called Distribution-Level Memory Recall (DMR). This method helps machines remember old knowledge better and not get mixed up between old and new information. They also address another problem called multimodal imbalance, which makes it harder for machines to learn from different types of data. The authors use several techniques, including Inter-modal Guidance and Intra-modal Mining (IGIM) and Incremental Mixup Feature Enhancement (IMFE), to help machines learn better without forgetting. |
Keywords
» Artificial intelligence » Classification » Continual learning » Mixture model » Multi modal » Recall
