Summary of Desparsify: Adversarial Attack Against Token Sparsification Mechanisms in Vision Transformers, by Oryan Yehezkel et al.
DeSparsify: Adversarial Attack Against Token Sparsification Mechanisms in Vision Transformers
by Oryan Yehezkel, Alon Zolfi, Amit Baras, Yuval Elovici, Asaf Shabtai
First submitted to arxiv on: 4 Feb 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper focuses on vision transformers’ high computational requirements, which grow quadratically with the number of tokens used. Token sparsification mechanisms are proposed to address this issue by discarding uninformative tokens from the computation pipeline, improving efficiency. However, these mechanisms are vulnerable to carefully crafted adversarial examples that can fool the sparsification mechanism, resulting in worst-case performance. To combat this threat, the authors introduce DeSparsify, an attack targeting the availability of vision transformers using token sparsification mechanisms. The attack aims to exhaust operating system resources while maintaining stealthiness. Evaluation demonstrates the attack’s effectiveness on three token sparsification mechanisms and examines transferability between them as well as its impact on GPU resources. To mitigate the attack’s impact, various countermeasures are proposed. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper talks about a problem with special computers that can see images (vision transformers). These computers need to process lots of information, but it takes too long. Some people have found ways to make them work faster by removing unimportant parts. However, someone might create fake images that could trick these computers into not working properly. To stop this from happening, the researchers created a special attack called DeSparsify. This attack makes the computer use up all its resources, but it does it quietly. They tested this attack on three different ways to make the computer work faster and found out how well it works and what kind of damage it can do. |
Keywords
* Artificial intelligence * Token * Transferability
