Summary of Preserving Silent Features For Domain Generalization, by Chujie Zhao et al.
Preserving Silent Features for Domain Generalization
by Chujie Zhao, Tianren Zhang, Feng Chen
First submitted to arxiv on: 6 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores domain generalization (DG), which aims to improve a model’s ability to generalize across unseen test domains after training on several known domains. Previous work has shown that self-supervised contrastive pre-training can enhance robustness, but this study finds that self-supervised models don’t outperform supervised models in the DG setting. The authors argue that self-supervised models extract richer intra-class features, which are then suppressed during fine-tuning. These “silent features” may contain generalizable information. They propose a method, STEP (Silent Feature Preservation), to preserve these silent features and improve self-supervised model performance. Results show that STEP achieves state-of-the-art performance on standard DG benchmarks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how well models can adapt to new situations after being trained on different types of data. Right now, we’re trying to make sure models are good at understanding things they haven’t seen before. We’ve found that some ways of training models don’t work as well as others when it comes to this task. The main idea is that when we train models using a certain method, they learn to focus on the most important details. But then, when we try to use them for something new, they forget those important details. This paper shows how we can keep those details and make our models even better. |
Keywords
* Artificial intelligence * Domain generalization * Fine tuning * Self supervised * Supervised
