Summary of Adversarial Training Via Adaptive Knowledge Amalgamation Of An Ensemble Of Teachers, by Shayan Mohajer Hamidi et al.
Adversarial Training via Adaptive Knowledge Amalgamation of an Ensemble of Teachers
by Shayan Mohajer Hamidi, Linfeng Ye
First submitted to arxiv on: 22 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces Adversarial Training via Adaptive Knowledge Amalgamation of an Ensemble of Teachers (AT-AKA), a novel method to train robust deep neural networks (DNNs) against adversarial attacks. AT-AKA addresses two shortcomings of traditional Adversarial Training (AT): the interdependence between robustness and model size, making it challenging to achieve robustness in smaller models; and poor generalization of adversarial samples, leaving DNNs vulnerable to unforeseen attack types. The proposed method generates diverse adversarial samples for an ensemble of teachers, which are then adapted to train a generalized-robust student. Experimental results demonstrate the superior efficacy of AT-AKA over existing AT methods and adversarial robustness distillation techniques against cutting-edge attacks, including AutoAttack. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper introduces a new way to make artificial intelligence (AI) models more secure against attacks. Current methods for making AI models robust have two problems: they only work well with big models, and the fake examples used to train them are not very good at generalizing. The new method, called AT-AKA, addresses these issues by using a team of teachers to create diverse fake examples that can help train a more robust student model. In experiments, this approach performed better than other methods against advanced attacks. |
Keywords
» Artificial intelligence » Distillation » Generalization » Student model
