Loading Now

Summary of Learning Equi-angular Representations For Online Continual Learning, by Minhyuk Seo et al.


Learning Equi-angular Representations for Online Continual Learning

by Minhyuk Seo, Hyunseo Koh, Wonje Jeung, Minjae Lee, San Kim, Hankook Lee, Sungjun Cho, Sungik Choi, Hyunwoo Kim, Jonghyun Choi

First submitted to arxiv on: 2 Apr 2024

Categories

  • Main: Computer Vision and Pattern Recognition (cs.CV)
  • Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

     Abstract of paper      PDF of paper


GrooveSquid.com Paper Summaries

GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!

Summary difficulty Written by Summary
High Paper authors High Difficulty Summary
Read the original abstract here
Medium GrooveSquid.com (original content) Medium Difficulty Summary
The paper proposes an efficient online continual learning method using the neural collapse phenomenon. The method induces neural collapse to form a simplex equiangular tight frame (ETF) structure in the representation space, allowing the continuously learned model with single-epoch training to better fit streamed data. This is achieved through preparatory data training and residual correction in the representation space. Empirical validations on CIFAR-10/100, TinyImageNet, ImageNet-200, and ImageNet-1K show that the proposed method outperforms state-of-the-art methods by a noticeable margin in various online continual learning scenarios.
Low GrooveSquid.com (original content) Low Difficulty Summary
The paper is about finding a way to make machines learn new things without forgetting old ones. Right now, this process doesn’t work very well because machines are only trained for a short time before being updated. The authors came up with an idea to use something called “neural collapse” to help the machine learn better. They tested their method on several types of data and found that it worked much better than other methods.

Keywords

» Artificial intelligence  » Continual learning