Loading Now

Summary of Continuous Autoregressive Models with Noise Augmentation Avoid Error Accumulation, by Marco Pasini et al.


Continuous Autoregressive Models with Noise Augmentation Avoid Error Accumulation

by Marco Pasini, Javier Nistal, Stefan Lattner, George Fazekas

First submitted to arxiv on: 27 Nov 2024

Categories

  • Main: Machine Learning (cs.LG)
  • Secondary: Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)

     Abstract of paper      PDF of paper


GrooveSquid.com Paper Summaries

GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!

Summary difficulty Written by Summary
High Paper authors High Difficulty Summary
Read the original abstract here
Medium GrooveSquid.com (original content) Medium Difficulty Summary
The paper proposes a novel approach to address the issue of error accumulation during inference in Continuous Autoregressive Models (CAMs) used for generating sequences of continuous embeddings. The authors introduce a method that injects random noise into input embeddings during training, making the model robust against varying error levels at inference. Additionally, they develop an inference procedure that introduces low-level noise to further reduce error accumulation. Experimental results on musical audio generation show that CAM outperforms existing autoregressive and non-autoregressive approaches while preserving audio quality over extended sequences.
Low GrooveSquid.com (original content) Low Difficulty Summary
This paper is about a new way to make models better at generating music and other continuous sounds. The problem is that the model gets worse as it goes longer, because small mistakes add up. To fix this, the authors added some randomness during training and then used that same randomness when generating the sound. This makes the model much better at making long sequences of music without getting worse.

Keywords

» Artificial intelligence  » Autoregressive  » Inference