Summary of Gaf: Gaussian Avatar Reconstruction From Monocular Videos Via Multi-view Diffusion, by Jiapeng Tang et al.
GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion
by Jiapeng Tang, Davide Davoli, Tobias Kirschstein, Liam Schoneveld, Matthias Niessner
First submitted to arxiv on: 13 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Graphics (cs.GR)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed approach for reconstructing animatable 3D Gaussian avatars from monocular videos uses a multi-view head diffusion model to fill in missing regions and ensure view consistency. The model is conditioned on VAE features extracted from the input image to preserve facial identity and appearance. To refine the denoised latent, latent upsampling is applied before decoding into an image. The method outperforms previous state-of-the-art methods in novel view synthesis by a 5.34% higher SSIM score on the NeRSemble dataset. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary We can reconstruct 3D avatars from smartphone videos! This new way of doing it uses special computer tricks to fill in missing parts and make sure the avatar looks right from different angles. It even keeps the person’s face looking like them! The method is really good at making the avatar look real, beating other methods by a lot. |
Keywords
» Artificial intelligence » Diffusion model
