Summary of Adversarial Robustness Overestimation and Instability in Trades, by Jonathan Weiping Li et al.
Adversarial Robustness Overestimation and Instability in TRADES
by Jonathan Weiping Li, Ren-Wei Liang, Cheng-Han Yeh, Cheng-Chang Tsai, Kuanchun Yu, Chun-Shien Lu, Shang-Tse Chen
First submitted to arxiv on: 10 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper examines probabilistic robustness overestimation in TRADES, a prominent adversarial training method. It reveals that TRADES can sometimes yield disproportionately high PGD validation accuracy compared to AutoAttack testing accuracy in multiclass classification tasks. The study highlights a significant overestimation of robustness for these instances, potentially linked to gradient masking. The authors analyze parameters contributing to unstable models that lead to overestimation and find associations with smaller batch sizes, lower beta values, larger learning rates, and higher class complexity. They identify the underlying cause as gradient masking and provide insights into it. The paper also explores metrics such as First-Order Stationary Condition (FOSC), inner-maximization, and gradient information to understand the phenomenon. Experiments show that certain unstable training instances may return to a state without robust overestimation, inspiring attempts at a solution. The authors recommend adjusting parameter settings to reduce instability or retraining when overestimation occurs. They also suggest incorporating Gaussian noise in inputs when the FOSC score exceeds a threshold to mitigate robustness overestimation. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how a popular way of training models to be more resistant to attacks (called TRADES) sometimes makes models seem much stronger than they really are. The researchers found that this “overestimation” is linked to something called gradient masking and that certain model settings can make it happen. They also looked at special metrics to understand why this overestimation happens. By looking at how well models do on different kinds of problems, the authors found that some models can get stuck in a state where they seem super strong but really aren’t. The paper suggests ways to fix this problem, like changing the way models are trained or adding extra noise to the data. This is important because it helps us understand why our models might not be as strong as we think they are and how we can make them better. |
Keywords
* Artificial intelligence * Classification
