Summary of Improving Image Clustering with Artifacts Attenuation Via Inference-time Attention Engineering, by Kazumoto Nakamura et al.
Improving Image Clustering with Artifacts Attenuation via Inference-Time Attention Engineering
by Kazumoto Nakamura, Yuji Nozawa, Yu-Chieh Lin, Kengo Nakata, Youyang Ng
First submitted to arxiv on: 7 Oct 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed paper aims to enhance the performance of pre-trained Vision Transformer (ViT) models, particularly DINOv2, in image clustering tasks without requiring re-training or fine-tuning. To address high-norm artifacts anomalies appearing in multi-head attention patches as model size increases, an approach called Inference-Time Attention Engineering (ITAE) is proposed, which manipulates the attention function during inference to identify and attenuate disproportionately large values. ITAE demonstrates improved clustering accuracy on multiple datasets by showcasing more expressive features in latent space, highlighting its potential as a practical solution for reducing artifacts and improving model performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper aims to improve pre-trained Vision Transformer models without re-training or fine-tuning. The problem is that larger models have high-norm artifacts that affect their performance. To fix this, the authors propose Inference-Time Attention Engineering (ITAE). ITAE helps by finding and reducing these bad values during inference. This makes the model better at clustering images. It works well on different datasets. |
Keywords
» Artificial intelligence » Attention » Clustering » Fine tuning » Inference » Latent space » Multi head attention » Vision transformer » Vit
