Summary of Mobile Video Diffusion, by Haitam Ben Yahia and Denis Korzhenkov and Ioannis Lelekas and Amir Ghodrati and Amirhossein Habibian
Mobile Video Diffusion
by Haitam Ben Yahia, Denis Korzhenkov, Ioannis Lelekas, Amir Ghodrati, Amirhossein Habibian
First submitted to arxiv on: 10 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces the first mobile-optimized video diffusion model, MobileVD, which achieves impressive realism and controllability while being significantly more efficient than existing models. The authors reduce memory and computational cost by reducing frame resolution, incorporating multi-scale temporal representations, and introducing novel pruning schema to reduce channels and temporal blocks. They also employ adversarial finetuning to simplify the denoising process. Compared to Stable Video Diffusion (SVD), MobileVD is 523x more efficient while only slightly decreasing quality, making it suitable for mobile devices. The model generates latents for a 14x512x256 px clip in just 1.7 seconds on a Xiaomi-14 Pro. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper makes video diffusion models more accessible on mobile devices by creating a new model called MobileVD. This model is better at using less memory and processing power, making it faster to work with. The creators of the model used different techniques to make it more efficient, such as reducing the size of each frame and simplifying the way it gets rid of noise. Even though it’s not as good as some other models, MobileVD can still create realistic videos that are fun to watch. |
Keywords
» Artificial intelligence » Diffusion » Diffusion model » Pruning
