Summary of Confidence-aware Denoised Fine-tuning Of Off-the-shelf Models For Certified Robustness, by Suhyeok Jang et al.
Confidence-aware Denoised Fine-tuning of Off-the-shelf Models for Certified Robustness
by Suhyeok Jang, Seojin Kim, Jinwoo Shin, Jongheon Jeong
First submitted to arxiv on: 13 Nov 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The recent surge in off-the-shelf classifiers has led to a reliance on large pre-trained models. However, these models are often vulnerable to adversarial attacks due to their training on clean data. Despite this vulnerability, they remain valuable for their superior performance and transferability. To provide post-hoc robustness, the denoised smoothing method was proposed, leveraging a denoiser model in front of the classifier. While effective, it introduces hallucination and distribution shift, leading to sub-optimal robustness. This paper introduces Fine-Tuning with Confidence-Aware Denoised Image Selection (FT-CADIS), a novel fine-tuning scheme that enhances certified robustness by identifying and handling hallucinated images during denoised smoothing. The confidence-aware training objective improves stability and allows for fine-tuning using beneficial images, updating only a small fraction of parameters. Experimental results demonstrate state-of-the-art certified robustness among denoised smoothing methods across various benchmarks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Off-the-shelf classifiers have revolutionized AI research, but they’re often vulnerable to attacks. Despite this, their performance and ability to adapt make them valuable. One way to improve them is by adding a “denoiser” that helps remove noise. However, this can create fake images that are misleading. The researchers in this paper found a way to fix this problem by creating a new method called Fine-Tuning with Confidence-Aware Denoised Image Selection (FT-CADIS). This method uses the confidence level of the classifier to decide which images are real and which are fake, and it only updates a small part of the model. The results show that this method is the best way to make off-the-shelf classifiers more robust. |
Keywords
» Artificial intelligence » Fine tuning » Hallucination » Transferability
