Summary of Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Models, by Giannis Daras et al.
Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Models
by Giannis Daras, Weili Nie, Karsten Kreis, Alex Dimakis, Morteza Mardani, Nikola Borislavov Kovachki, Arash Vahdat
First submitted to arxiv on: 21 Oct 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper addresses the limitations of using image models for solving inverse video problems, which often result in flickering, texture-sticking, and temporal inconsistency in generated videos. The authors propose a novel approach by viewing frames as continuous functions in 2D space and videos as sequences of continuous warping transformations between frames. This perspective enables training function space diffusion models on images to solve temporally correlated inverse problems. The method requires equivariance with respect to spatial transformations, which is ensured through post-hoc test-time guidance towards self-equivariant solutions. The authors demonstrate the effectiveness of their approach for video inpainting and 8x video super-resolution, outperforming existing techniques based on noise transformations. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper fixes a problem in using image models to solve video problems. When we try to use these models to make new videos, they often look bad because they don’t match the original video’s movement or texture. The authors came up with a new way to think about videos as continuous movements between frames, which lets them train special models that can fix these issues. This helps create better results for things like filling in missing parts of a video and making old videos clearer. |
Keywords
» Artificial intelligence » Diffusion » Super resolution
