Summary of Towards Effective Usage Of Human-centric Priors in Diffusion Models For Text-based Human Image Generation, by Junyan Wang et al.
Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generation
by Junyan Wang, Zhenhong Sun, Zhiyu Tan, Xuanbai Chen, Weihua Chen, Hao Li, Cheng Zhang, Yang Song
First submitted to arxiv on: 8 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel approach to improve the accuracy of human image generation in text-to-image diffusion models. The existing methods fine-tune the model using extra images or additional controls, but this paper integrates human-centric priors directly into the fine-tuning stage. A human-centric alignment loss is introduced to strengthen human-related information from textual prompts within cross-attention maps. To ensure semantic detail richness and human structural accuracy during fine-tuning, scale-aware and step-wise constraints are applied within the diffusion process. The proposed method outperforms state-of-the-art text-to-image models in generating high-quality human images based on user-written prompts. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making computers better at creating realistic pictures of people. Right now, these computer programs often get things wrong and create weird or unnatural-looking bodies. The researchers tried to fix this by adding special instructions to the program that tell it what a human looks like. They tested their idea and found that it works much better than other approaches. This means we can use computers to generate more realistic pictures of people, which could be useful for things like creating artwork or making movies. |
Keywords
* Artificial intelligence * Alignment * Cross attention * Diffusion * Fine tuning * Image generation
