Summary of The Surprising Harmfulness Of Benign Overfitting For Adversarial Robustness, by Yifan Hao et al.
The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness
by Yifan Hao, Tong Zhang
First submitted to arxiv on: 19 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Cryptography and Security (cs.CR); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the generalization capabilities of large machine learning models trained on noisy data. Despite previous findings that benign overfitting can be harmless when out-of-sample data is unaffected, this study reveals a surprising consequence: even if the ground truth is robust to adversarial examples and the model is benign in terms of standard risk, benign overfitting can still lead to adverse vulnerability when faced with adversarially manipulated data. The authors prove that min-norm estimators in overparameterized linear models always result in adversarial vulnerability in the benign overfitting setting, and demonstrate an asymptotic trade-off between standard and adversarial risks of ridge regression estimators. They also explore these findings in the context of two-layer neural tangent kernel (NTK) models under lazy training regimes, which aligns with empirical observations in deep neural networks. This research provides theoretical insights into a puzzling phenomenon where robust target functions lead to non-robust models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large machine learning models can be very good at predicting things, but this paper shows that even if the data is noisy and the model is doing well on its own data, it can still have trouble with new, fake data. This is because the model is not good at dealing with special kinds of bad data called “adversarial examples”. The authors found some surprising results about how models can be too good at fitting noise in the data, even if that’s okay for normal data. But when it comes to bad data, this overfitting can make things worse. This is important because it helps us understand why sometimes our models don’t work well with new data, even if they seem fine on their own data. |
Keywords
* Artificial intelligence * Generalization * Machine learning * Overfitting * Regression
