Summary of Bdetclip: Multimodal Prompting Contrastive Test-time Backdoor Detection, by Yuwei Niu et al.
BDetCLIP: Multimodal Prompting Contrastive Test-Time Backdoor Detection
by Yuwei Niu, Shuo He, Qi Wei, Zongyu Wu, Feng Liu, Lei Feng
First submitted to arxiv on: 24 May 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel computationally efficient method for detecting backdoored CLIP models at the inference stage. Recent research has shown that multimodal contrastive learning can be vulnerable to backdoor attacks, which involve inserting triggers into pre-trained models. Existing defense methods focus on either the pre-training or fine-tuning stages, but these approaches require numerous parameter updates and are computationally expensive. The proposed method, called BDetCLIP, uses a language model (e.g., GPT-4) to produce class-related description texts and perturbed random texts, which are then used to compute cosine similarity with images. This distribution difference is used as the criterion to detect backdoor samples. Experimental results show that BDetCLIP outperforms state-of-the-art methods in both effectiveness and efficiency. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper tries to fix a problem with AI models called “backdoored CLIP”. These models are trained on fake data, which can make them do things they shouldn’t when used later. The researchers found that these backdoor attacks can be very successful. They also found that the way we usually defend against this kind of attack is too slow and uses a lot of computing power. To solve this problem, they came up with a new method called BDetCLIP. This method uses special texts to trick the AI model into revealing when it’s being attacked. The results show that their method works better than others at catching backdoor attacks while also being faster. |
Keywords
» Artificial intelligence » Cosine similarity » Fine tuning » Gpt » Inference » Language model
