Summary of Robust Gaussian Processes Via Relevance Pursuit, by Sebastian Ament et al.
Robust Gaussian Processes via Relevance Pursuit
by Sebastian Ament, Elizabeth Santorella, David Eriksson, Ben Letham, Maximilian Balandat, Eytan Bakshy
First submitted to arxiv on: 31 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Optimization and Control (math.OC); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Gaussian processes (GPs) are widely used due to their flexibility, data efficiency, and well-calibrated uncertainty estimates. However, standard GP models assume homoskedastic Gaussian noise, which may not be suitable for real-world applications with non-Gaussian corruptions. Researchers have proposed variants of GPs that are more robust to alternative noise models, but these come at a cost of reduced accuracy or increased computational requirements. This paper proposes a new GP model that infers data-point-specific noise levels using a sequential selection procedure maximizing the log marginal likelihood, known as relevance pursuit. The study shows that this model can be parameterized to ensure a strongly concave log marginal likelihood in the data-point-specific noise variances, which is rare in both robust regression objectives and GP marginal likelihoods. This property implies weak submodularity of the subset selection problem and proves approximation guarantees for the proposed algorithm. The paper evaluates the model’s performance on various tasks, including sparse corruptions of labels within or close to the function range. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Gaussian processes are a type of mathematical model that can learn patterns in data. But sometimes, real-world data has unexpected noise or errors. In this case, standard models don’t work well. Researchers have tried to fix this by creating new models that are more robust to these noises. This paper proposes a new way to do this by adjusting the noise levels for each data point based on how good the model is at predicting it. The study shows that this approach can be very effective and can even solve problems where the noise is sparse or close to the expected outcome. |
Keywords
» Artificial intelligence » Likelihood » Regression
