Summary of Robust Neural Processes For Noisy Data, by Chen Shapira et al.
Robust Neural Processes for Noisy Data
by Chen Shapira, Dan Rosenbaum
First submitted to arxiv on: 3 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Models that adapt their predictions based on given contexts, known as in-context learning, have become widespread in recent years. We investigate the behavior of such models when data is contaminated by noise using the Neural Processes (NP) framework, which learns a distribution over functions and makes predictions based on context points. Our findings reveal that models performing well on clean data differ from those performing well on noisy data. Specifically, attention-processing models are severely affected by noise, leading to in-context overfitting. To address this issue, we propose a simple method for training NP models that boosts their robustness to noisy data. Experiments on 1D functions and 2D image datasets demonstrate that our approach outperforms all other NP models across various noise levels. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary In-context learning is when AI models adapt their predictions based on given contexts. But what happens when the data has noise? We looked at how Neural Processes (NP) models work in this situation. NPs learn a distribution over functions and make predictions based on context points. Our results show that models that do well with clean data don’t necessarily do well with noisy data. In fact, attention-processing models are really bad at handling noise. To fix this, we came up with a simple way to train NP models so they’re more robust to noisy data. We tested our method on 1D functions and 2D image datasets and found that it works better than other NPs for all levels of noise. |
Keywords
» Artificial intelligence » Attention » Overfitting
