Summary of Sse-sam: Balancing Head and Tail Classes Gradually Through Stage-wise Sam, by Xingyu Lyu et al.
SSE-SAM: Balancing Head and Tail Classes Gradually through Stage-Wise SAM
by Xingyu Lyu, Qianqian Xu, Zhiyong Yang, Shaojie Lyu, Qingming Huang
First submitted to arxiv on: 18 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel approach called Stage-wise Saddle Escaping SAM (SSE-SAM) for improving the generalization of deep learning models in long-tailed datasets. Traditional methods like Sharpness-Aware Minimization (SAM) tend to overfit on tail classes, while Imbalanced SAM (ImbSAM) enhances the smoothness of the loss function for tail classes but neglects head classes. The authors argue that a careful balance between head and tail classes is necessary for effective generalization. They show that neither SAM nor ImbSAM alone can achieve this balance, and propose SSE-SAM as a phased approach that first avoids saddle points in head-class losses and then focuses on tail-classes to help them escape saddle points. The authors demonstrate the effectiveness of SSE-SAM in escaping saddles both for head and tail classes, leading to performance improvements. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary In simple terms, this paper is about making machine learning models work better when there are many more examples of some types than others. This is a common problem in real-world datasets, but traditional methods don’t always do well in these situations. The authors propose a new approach that tries to balance the training of different classes by first focusing on the majority class and then switching to the minority class. They show that this approach works better than previous methods and improves performance. |
Keywords
» Artificial intelligence » Deep learning » Generalization » Loss function » Machine learning » Sam
