Summary of Peeraid: Improving Adversarial Distillation From a Specialized Peer Tutor, by Jaewon Jung et al.
PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor
by Jaewon Jung, Hongsun Jang, Jaeyong Song, Jinho Lee
First submitted to arxiv on: 11 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Adversarial robustness of neural networks is a pressing concern in security-critical domains. Adversarial distillation aims to transfer the robustness of a teacher network to a smaller student network. Previous methods pretrain teacher networks against adversarial examples designed for themselves, but this approach degrades when faced with unseen transferred attacks targeting the student network’s parameters. Our proposed PeerAiD addresses this limitation by training a peer network alongside the student network to specialize in defending it. We demonstrate that these peer networks surpass the robustness of pre-trained robust teacher models against adversarial examples designed for the student network. In experiments, PeerAiD achieves up to 1.66%p improvement in AutoAttack accuracy and 4.72%p increase in natural accuracy with ResNet-18 on TinyImageNet dataset. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making sure that artificial intelligence (AI) systems are not easily tricked into making mistakes when they’re really important, like in security situations. One way to do this is by “distilling” the strong traits of a bigger AI model into a smaller one. The problem is that current methods only make the big AI model stronger against its own weaknesses, but don’t help it defend against new and unexpected attacks. Our new method, PeerAiD, trains two AI models together to work better as a team and protect each other from these kinds of attacks. |
Keywords
* Artificial intelligence * Distillation * Resnet
