Summary of Criticality Leveraged Adversarial Training (clat) For Boosted Performance Via Parameter Efficiency, by Bhavna Gopal et al.
Criticality Leveraged Adversarial Training (CLAT) for Boosted Performance via Parameter Efficiency
by Bhavna Gopal, Huanrui Yang, Jingyang Zhang, Mark Horton, Yiran Chen
First submitted to arxiv on: 19 Aug 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach called CLAT (Critical Layer Adversarial Training) is introduced to enhance neural network robustness against adversarial attacks while minimizing overfitting. Traditional adversarial training methods can suffer from increased generalization errors on clean data and overfitting, which hinders their effectiveness. CLAT addresses this issue by identifying the most critical layers in the model that are learning non-robust features and fine-tuning them while freezing the remaining layers to preserve robustness. This technique is shown to be highly effective, achieving a significant reduction of approximately 95% in trainable parameters and an improvement of over 2% in adversarial robustness compared to baseline methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A new way to make neural networks more resilient against fake data called CLAT has been discovered. Right now, when we train these networks with fake data, they can become too good at recognizing this fake data and forget how to recognize real data. CLAT helps by identifying the most important parts of the network that are learning from the fake data and making those parts better while leaving the rest alone. This makes the whole network more robust against both fake and real data. |
Keywords
» Artificial intelligence » Fine tuning » Generalization » Neural network » Overfitting
