Summary of Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature Of Attention, by Susung Hong
Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention
by Susung Hong
First submitted to arxiv on: 1 Aug 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Conditional diffusion models have achieved remarkable success in visual content generation, largely due to classifier-free guidance (CFG). However, recent attempts to extend guidance to unconditional models rely on heuristic techniques, resulting in suboptimal quality and unintended effects. This work proposes Smoothed Energy Guidance (SEG), a novel training- and condition-free approach that leverages the energy-based perspective of self-attention to enhance image generation. SEG reduces the curvature of the energy landscape by adjusting the Gaussian kernel parameter while keeping the guidance scale fixed. Additionally, it presents a query blurring method that blurs attention weights without increasing complexity. In experiments, SEG achieves a Pareto improvement in both quality and reduction of side effects. The code is available at https://github.com/SusungHong/SEG-SDXL. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper improves how computers generate images. They used a new way to make the computer’s attention focus better. This means the generated images will be more realistic. They tested this method and it worked well, producing good-quality images with fewer unwanted effects. The code for this method is available online. |
Keywords
* Artificial intelligence * Attention * Diffusion * Image generation * Self attention
