Summary of Improving Sam Requires Rethinking Its Optimization Formulation, by Wanyun Xie et al.
Improving SAM Requires Rethinking its Optimization Formulation
by Wanyun Xie, Fabian Latorre, Kimon Antonakopoulos, Thomas Pethick, Volkan Cevher
First submitted to arxiv on: 17 Jul 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper revisits the Sharpness-Aware Minimization (SAM) algorithm, originally framed as a zero-sum game between network weights and perturbations. The authors propose a reformulation of SAM using the 0-1 loss, leading to a novel bilevel optimization problem dubbed BiSAM. This new formulation constructs stronger perturbations and achieves improved performance compared to original SAM and variants, with similar computational complexity. The code is available at https://github.com/LIONS-EPFL/BiSAM. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper changes how we use an algorithm called Sharpness-Aware Minimization (SAM) to make machine learning models more robust. Instead of using a zero-sum game, the authors suggest using a different way to measure how well or badly the model performs. This new approach, called BiSAM, makes the model’s predictions better and is just as efficient to use. |
Keywords
» Artificial intelligence » Machine learning » Optimization » Sam
