Summary of Disrupting Diffusion: Token-level Attention Erasure Attack Against Diffusion-based Customization, by Yisu Liu et al.
Disrupting Diffusion: Token-Level Attention Erasure Attack against Diffusion-based Customization
by Yisu Liu, Jinyang An, Wanqian Zhang, Dayan Wu, Jingzi Gu, Zheng Lin, Weiping Wang
First submitted to arxiv on: 31 May 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed DisDiff (Disrupting Diffusion) method is a novel adversarial attack approach that disrupts the outputs of diffusion-based customization models, aiming to protect users from malicious misuse. By empirically understanding the role of cross-attention in guiding image generation, DisDiff introduces the Cross-Attention Erasure module to explicitly erase attention maps and disrupt text guidance. Additionally, it analyzes the impact of sampling processes on Projected Gradient Descent (PGD) attacks, introducing a Merit Sampling Scheduler to adaptively update perturbation amplitudes. DisDiff outperforms state-of-the-art methods by 12.75% in FDFR scores and 7.25% in ISM scores across two facial benchmarks and prompts. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary DisDiff is a new way to stop people from misusing AI models that make personalized images. These models are great for things like changing your appearance, but they can also be used to create fake images that could hurt people’s privacy. DisDiff tries to solve this problem by making it harder for malicious users to create these fake images. It does this by understanding how the model works and then finding ways to disrupt its output. The method is tested on facial recognition benchmarks and shows better results than other similar methods. |
Keywords
» Artificial intelligence » Attention » Cross attention » Diffusion » Gradient descent » Image generation
