Summary of Derivation Of Closed Form Of Expected Improvement For Gaussian Process Trained on Log-transformed Objective, by Shuhei Watanabe
Derivation of Closed Form of Expected Improvement for Gaussian Process Trained on Log-Transformed Objective
by Shuhei Watanabe
First submitted to arxiv on: 27 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper proposes a novel approach to improve the Expected Improvement (EI) acquisition function in Bayesian optimization, which is widely used but often challenging to enhance due to its sensitivity to numerical precision. The authors build upon previous work by Hutter et al. (2009), who trained Gaussian processes on log-transformed objective functions to improve predictive accuracy. This paper provides a friendly derivation of the EI proposition, offering a new perspective for improving EI’s performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making a special kind of computer algorithm better at finding the best solution. Right now, this algorithm is called Expected Improvement (EI) and it’s really good, but it has one big problem: it gets stuck if you’re not careful with the numbers. Some smart people figured out how to make EI work better by changing how it looks at things, and now they’re sharing their secret recipe to make EI even more powerful. |
Keywords
* Artificial intelligence * Optimization * Precision
