Summary of Conserve-update-revise to Cure Generalization and Robustness Trade-off in Adversarial Training, by Shruthi Gowda et al.
Conserve-Update-Revise to Cure Generalization and Robustness Trade-off in Adversarial Training
by Shruthi Gowda, Bahram Zonooz, Elahe Arani
First submitted to arxiv on: 26 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the phenomenon where neural networks improve their robustness against adversarial attacks through adversarial training, but at the cost of standard generalization. The authors examine layer-wise learning capabilities during this transition and find that selectively updating specific layers while preserving others can enhance the network’s learning capacity. They propose a novel framework called CURE (Conservation, Updating, and Revision of weights) that leverages a gradient prominence criterion to perform selective conservation, updating, and revision of weights. This framework is designed to be dataset-agnostic and architecture-agnostic, making it applicable across various scenarios. It effectively tackles memorization and overfitting issues, improving the trade-off between robustness and generalization. The authors also provide insights into the mechanisms of selective adversarial training and open up a promising avenue for future research. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how neural networks can become better at fighting fake information by being trained on fake data. Right now, these networks are getting better at recognizing and fighting this fake information, but it’s coming at the cost of not doing as well with normal information. The researchers looked at how different layers of the network are learning during this process and found that some layers can be “frozen” while others are updated, which makes the network learn faster and better. They also came up with a new way to train these networks, called CURE, which helps them avoid mistakes like memorizing specific examples instead of following rules. |
Keywords
* Artificial intelligence * Generalization * Overfitting
