Summary of Tcan: Animating Human Images with Temporally Consistent Pose Guidance Using Diffusion Models, by Jeongho Kim et al.
TCAN: Animating Human Images with Temporally Consistent Pose Guidance using Diffusion Models
by Jeongho Kim, Min-Jung Kim, Junsoo Lee, Jaegul Choo
First submitted to arxiv on: 12 Jul 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel pose-driven human image animation method called TCAN, which achieves temporally consistent and robust animation despite erroneous poses. Unlike previous approaches that fine-tune ControlNet for pose-image-caption pairs, TCAN utilizes the pre-trained model without fine-tuning to leverage its extensive knowledge. To enhance robustness against outliers of the pose detector, an additional temporal layer is introduced to the ControlNet. Experimental results demonstrate promising performance in video synthesis tasks with various poses, including chibi. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a new way to make realistic videos of people doing different things. The method uses information from pre-trained models and special layers to keep the animation consistent over time. It also helps the model ignore incorrect pose information. The results show that this method can create great-looking videos with people in different poses, like cartoons. |
Keywords
» Artificial intelligence » Fine tuning
