Summary of From Feature Visualization to Visual Circuits: Effect Of Adversarial Model Manipulation, by Geraldin Nanfack et al.
From Feature Visualization to Visual Circuits: Effect of Adversarial Model Manipulation
by Geraldin Nanfack, Michael Eickenberg, Eugene Belilovsky
First submitted to arxiv on: 3 Jun 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper tackles the challenge of understanding large-scale deep neural networks by analyzing human-understandable subgraphs called circuits in vision-pretrained models. Existing works have studied the stability of feature visualization under adversarial attacks, but this study proposes a novel attack called ProxPulse that manipulates both node and edge features. Surprisingly, visual circuits show some robustness to ProxPulse, leading to the introduction of a new attack that reveals their lack of robustness. The effectiveness of these attacks is validated using pre-trained AlexNet and ResNet-50 models on ImageNet. This work contributes to the emerging field of mechanistic interpretability in deep learning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study tries to figure out how big artificial intelligence systems called neural networks work by looking at smaller parts inside them. People have already studied how these systems can be tricked, but this research proposes a new way to do it called ProxPulse. Interestingly, the small parts of the system are kind of resistant to being tricked in this way. The researchers then developed a new attack that shows how easily these small parts can be manipulated. They tested their attacks on two big models and found that they worked. |
Keywords
» Artificial intelligence » Deep learning » Resnet
