Summary of Acdc: Autoregressive Coherent Multimodal Generation Using Diffusion Correction, by Hyungjin Chung et al.
ACDC: Autoregressive Coherent Multimodal Generation using Diffusion Correction
by Hyungjin Chung, Dohun Lee, Jong Chul Ye
First submitted to arxiv on: 7 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces Autoregressive Coherent multimodal generation with Diffusion Correction (ACDC), a zero-shot approach that combines the strengths of autoregressive models (ARMs) and diffusion models (DMs). ACDC leverages ARMs for global context generation and memory-conditioned DMs for local correction, ensuring high-quality outputs by correcting artifacts in generated multimodal tokens. The proposed memory module based on large language models (LLMs) dynamically adjusts the conditioning texts for the DMs, preserving crucial global context information. Experimental results on multimodal tasks, including coherent multi-frame story generation and autoregressive video generation, demonstrate that ACDC effectively mitigates error accumulation and significantly enhances output quality, achieving superior performance while remaining agnostic to specific ARM and DM architectures. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary ACDC is a new way to generate images and videos. It uses two different models: autoregressive models (ARMs) and diffusion models (DMs). ARMs are good at generating long sequences of data, like videos, but they can make mistakes that add up over time. DMs are better at generating high-quality local contexts, like a single image. ACDC combines the strengths of both models to create more accurate and realistic images and videos. |
Keywords
» Artificial intelligence » Autoregressive » Diffusion » Zero shot
