Summary of Approximation-aware Bayesian Optimization, by Natalie Maus et al.
Approximation-Aware Bayesian Optimization
by Natalie Maus, Kyurae Kim, Geoff Pleiss, David Eriksson, John P. Cunningham, Jacob R. Gardner
First submitted to arxiv on: 6 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel approach to Bayesian optimization (BO) that addresses the limitations of existing methods, such as sparse variational Gaussian processes (SVGPs), which can slow down the optimization process. The authors modify SVGPs to better align with the goals of BO, targeting informed data acquisition rather than global posterior fidelity. This is achieved by unifying GP approximation and data acquisition into a joint optimization problem using the framework of utility-calibrated variational inference. The approach is applicable with any decision-theoretic acquisition function and is compatible with trust region methods like TuRBO. The authors also derive efficient joint objectives for the expected improvement and knowledge gradient acquisition functions in both standard and batch BO settings. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper improves Bayesian optimization by making it more efficient and effective. It does this by changing how Gaussian processes (GPs) are used to make predictions. Instead of just trying to get a good overall picture, the new approach focuses on making informed decisions about what data to collect next. This helps BO work better in high-dimensional spaces, where it’s harder to find good solutions. The method is flexible and can be used with different ways of choosing what data to collect. |
Keywords
* Artificial intelligence * Inference * Optimization
