Summary of Ccsi: Continual Class-specific Impression For Data-free Class Incremental Learning, by Sana Ayromlou et al.
CCSI: Continual Class-Specific Impression for Data-free Class Incremental Learning
by Sana Ayromlou, Teresa Tsang, Purang Abolmaesumi, Xiaoxiao Li
First submitted to arxiv on: 9 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed paper addresses the issue of diagnosing newly introduced disease types in clinical settings using deep learning-based classification methods. Traditional approaches require samples from all disease classes for offline training, which can be challenging and ineffective when dealing with novel diseases. Class incremental learning offers a solution by adapting a deep network trained on specific disease classes to handle new diseases. However, this approach often leads to catastrophic forgetting, where the performance of earlier classes decreases when adapting the model to new data. The proposed framework utilizes data synthesis on learned classes instead of storing previous samples, which poses practical concerns regarding privacy and storage regulations in healthcare. The key contributions include acquiring synthetic data known as Continual Class-Specific Impression (CCSI) for previously inaccessible trained classes and presenting a methodology to effectively utilize this data for updating networks when introducing new classes. The proposed framework acquires CCSI by employing data inversion over gradients of the trained classification model on previous classes, starting from the mean image of each class. This process utilizes continual normalization layers statistics as a regularizer in the pixel-wise optimization process. Subsequently, the network is updated by combining synthesized data with new class data and incorporating several losses, including intra-domain contrastive loss, margin loss, and cosine-normalized cross-entropy loss. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper proposes a novel framework for real-world clinical settings to diagnose newly introduced disease types using deep learning-based classification methods. This framework addresses the issue of catastrophic forgetting in class incremental learning by utilizing data synthesis on learned classes instead of storing previous samples. In essence, this paper is about developing a new approach to adapt machine learning models when introducing new diseases or classes without requiring storage of previous samples. The proposed method, Continual Class-Specific Impression (CCSI), uses synthetic data generated from gradients of the trained classification model and combines it with new class data for updating networks. |
Keywords
» Artificial intelligence » Classification » Contrastive loss » Cross entropy » Deep learning » Machine learning » Optimization » Synthetic data
