Summary of Morphable Diffusion: 3d-consistent Diffusion For Single-image Avatar Creation, by Xiyi Chen et al.
Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation
by Xiyi Chen, Marko Mihajlovic, Shaofei Wang, Sergey Prokudin, Siyu Tang
First submitted to arxiv on: 9 Jan 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A generative diffusion model can create photorealistic human avatars from a single image or text prompt, with advanced capabilities including facial expression and body pose control. To improve this capability, researchers integrated a 3D morphable model into the state-of-the-art multi-view-consistent diffusion approach. This integration led to enhanced performance on novel view synthesis tasks and enabled seamless control of facial expressions and body poses. The proposed framework is the first to create fully 3D-consistent, animatable, and photorealistic human avatars from a single image. Extensive evaluations demonstrate its advantages over existing state-of-the-art models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Generative diffusion models can now make realistic-looking people from just one picture or some words. The goal is to make these models better for creating avatars that look like real people and can be controlled to change their expressions and poses. To do this, the researchers combined two important ideas: a 3D model of a person’s face and body, and a way to generate new images using that information. This combination made the generated pictures much more realistic and allowed for better control over what they look like. |
Keywords
* Artificial intelligence * Diffusion * Diffusion model * Prompt
