Summary of Depth-guided Nerf Training Via Earth Mover’s Distance, by Anita Rau et al.
Depth-guided NeRF Training via Earth Mover’s Distance
by Anita Rau, Josiah Aklilu, F. Christopher Holsinger, Serena Yeung-Levy
First submitted to arxiv on: 19 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Neural Radiance Fields (NeRFs) are trained to minimize rendering loss, but the photometric loss lacks information to disambiguate between different geometries. Previous work incorporated depth supervision during NeRF training using pre-trained depth networks as pseudo-ground truth. However, these depth priors can be noisy and challenging to capture accurately. This paper proposes a novel approach to uncertainty in depth priors for NeRF supervision by using off-the-shelf pretrained diffusion models to predict depth and capture uncertainty during denoising. The authors suggest supervising the ray termination distance distribution with Earth Mover’s Distance instead of enforcing rendered depth to replicate the depth prior exactly through L2-loss. This novel approach outperforms all baselines on standard depth metrics by a large margin while maintaining performance on photometric measures. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about improving how Neural Radiance Fields (NeRFs) learn from images. NeRFs are like super-powerful cameras that can generate new images of the same scene from different viewpoints. But, sometimes they get confused and produce multiple possible geometries for the same image. To help them make better choices, previous methods used depth information to guide their learning. However, these depth guides can be noisy and hard to correct. This paper proposes a new way to use off-the-shelf algorithms to predict depth and capture uncertainty during training. By doing so, they improve the accuracy of the NeRFs in predicting the 3D scene from 2D images. |
Keywords
» Artificial intelligence » Diffusion
