Summary of Continuous Autoregressive Models with Noise Augmentation Avoid Error Accumulation, by Marco Pasini et al.
Continuous Autoregressive Models with Noise Augmentation Avoid Error Accumulation
by Marco Pasini, Javier Nistal, Stefan Lattner, George Fazekas
First submitted to arxiv on: 27 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel approach to address the issue of error accumulation during inference in Continuous Autoregressive Models (CAMs) used for generating sequences of continuous embeddings. The authors introduce a method that injects random noise into input embeddings during training, making the model robust against varying error levels at inference. Additionally, they develop an inference procedure that introduces low-level noise to further reduce error accumulation. Experimental results on musical audio generation show that CAM outperforms existing autoregressive and non-autoregressive approaches while preserving audio quality over extended sequences. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about a new way to make models better at generating music and other continuous sounds. The problem is that the model gets worse as it goes longer, because small mistakes add up. To fix this, the authors added some randomness during training and then used that same randomness when generating the sound. This makes the model much better at making long sequences of music without getting worse. |
Keywords
» Artificial intelligence » Autoregressive » Inference
