Summary of Efficient Test-time Model Adaptation Without Forgetting, by Shuaicheng Niu and Jiaxiang Wu and Yifan Zhang and Yaofo Chen and Shijian Zheng and Peilin Zhao and Mingkui Tan
Efficient Test-Time Model Adaptation without Forgetting
by Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, Mingkui Tan
First submitted to arxiv on: 6 Apr 2022
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel approach to test-time adaptation (TTA) for deep models, addressing two practical challenges: high prediction costs and performance degradation on in-distribution data after TTA. The authors argue that not all test samples contribute equally to model adaptation and introduce an active sample selection criterion to identify reliable and non-redundant samples. They also propose a Fisher regularizer to constrain important model parameters from drastic changes, alleviating the forgetting issue. The method is evaluated on CIFAR-10-C, ImageNet-C, and ImageNet-R datasets, demonstrating its effectiveness in improving test performance while minimizing catastrophic forgetting. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps deep models adapt to new situations by picking the most important information and keeping what’s already working well. Currently, making models adjust to new data is expensive and can make them worse at recognizing things they already knew how to identify. The researchers found that not all new data is created equal – some bits are more important than others. They developed a way to figure out which bits are most useful and then updated the model just for those ones. This made the model better at recognizing new things without forgetting what it already knew. The method was tested on three different types of images and showed promising results. |
