Summary of Slowing Down Forgetting in Continual Learning, by Pascal Janetzky et al.
Slowing Down Forgetting in Continual Learning
by Pascal Janetzky, Tobias Schlagenhauf, Stefan Feuerriegel
First submitted to arxiv on: 11 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel framework called ReCL for continual learning (CL) that slows down catastrophic forgetting. The authors exploit an implicit bias of gradient-based neural networks, converging to margin maximization points, allowing them to reconstruct old data from previous tasks. This is combined with current training data, making the framework flexible and applicable on top of existing CL methods. Experiments demonstrate large performance gains across various scenarios (class incremental and domain incremental learning), datasets (MNIST, CIFAR10, TinyImagenet), and network architectures. The authors claim their framework is the first to address catastrophic forgetting by leveraging models in CL as their own memory buffers. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us learn better over time without losing what we already know. It’s like a special way to remember old things while learning new ones. The method, called ReCL, uses something about how computers learn that helps them keep track of old information. This means it can be used with other ways of learning new things, and it works well across different situations and types of data. The results show big improvements in performance. |
Keywords
» Artificial intelligence » Continual learning
