Summary of Downstream Transfer Attack: Adversarial Attacks on Downstream Models with Pre-trained Vision Transformers, by Weijie Zheng et al.
Downstream Transfer Attack: Adversarial Attacks on Downstream Models with Pre-trained Vision Transformers
by Weijie Zheng, Xingjun Ma, Hanxun Huang, Zuxuan Wu, Yu-Gang Jiang
First submitted to arxiv on: 3 Aug 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the vulnerability of pre-trained vision transformers (ViTs) to sample-wise transfer attacks. The researchers propose a novel attack method called Downstream Transfer Attack (DTA), which leverages a pre-trained ViT model to craft adversarial examples and then applies them to fine-tuned models on downstream datasets. DTA identifies vulnerable layers in the pre-trained model using a cosine similarity loss, allowing it to generate highly transferable attacks. The study demonstrates the effectiveness of DTA across 10 diverse downstream datasets, achieving an average attack success rate (ASR) exceeding 90%, outperforming existing methods by a significant margin. When used with adversarial training, the generated adversarial examples improve the model’s robustness to different downstream transfer attacks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research looks at how well pre-trained artificial intelligence models can be tricked into making mistakes. These models are called vision transformers, or ViTs for short. The researchers found that these models can be fooled by adding small changes to the input images. They created a new way to do this called Downstream Transfer Attack (DTA). DTA is very good at finding and exploiting weaknesses in the pre-trained models. In tests with 10 different types of data, DTA was able to trick the models into making mistakes more than 90% of the time! When used together with special training, these tricks can actually make the models better at handling similar attacks in the future. |
Keywords
» Artificial intelligence » Cosine similarity » Vit
