Summary of Learning Equi-angular Representations For Online Continual Learning, by Minhyuk Seo et al.
Learning Equi-angular Representations for Online Continual Learning
by Minhyuk Seo, Hyunseo Koh, Wonje Jeung, Minjae Lee, San Kim, Hankook Lee, Sungjun Cho, Sungik Choi, Hyunwoo Kim, Jonghyun Choi
First submitted to arxiv on: 2 Apr 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes an efficient online continual learning method using the neural collapse phenomenon. The method induces neural collapse to form a simplex equiangular tight frame (ETF) structure in the representation space, allowing the continuously learned model with single-epoch training to better fit streamed data. This is achieved through preparatory data training and residual correction in the representation space. Empirical validations on CIFAR-10/100, TinyImageNet, ImageNet-200, and ImageNet-1K show that the proposed method outperforms state-of-the-art methods by a noticeable margin in various online continual learning scenarios. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about finding a way to make machines learn new things without forgetting old ones. Right now, this process doesn’t work very well because machines are only trained for a short time before being updated. The authors came up with an idea to use something called “neural collapse” to help the machine learn better. They tested their method on several types of data and found that it worked much better than other methods. |
Keywords
» Artificial intelligence » Continual learning
