Summary of High-dimensional Bayesian Optimization Via Random Projection Of Manifold Subspaces, by Quoc-anh Hoang Nguyen et al.
High-Dimensional Bayesian Optimization via Random Projection of Manifold Subspaces
by Quoc-Anh Hoang Nguyen, Hung Tran
First submitted to arxiv on: 21 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach is proposed for Bayesian Optimization (BO) in high dimensions by exploiting a new representation of the objective function. The method combines random linear projection to reduce dimensionality with representation learning of the nonlinear manifold. When geometry is available, a solution to exploit this geometry is presented. To mitigate overfitting, the feature mapping is trained in a geometry-aware semi-supervised manner using a neural network. This approach enables efficient optimization of BO’s acquisition function in the low-dimensional space and projecting back to the original high-dimensional space. Empirical results show that the algorithm outperforms other high-dimensional BO baselines on various synthetic functions and real applications. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A new way is found to make Bayesian Optimization work better when there are many variables. The method uses a combination of reducing the number of variables and learning how to map the important variables onto a simpler space. This helps by making it easier to find the best option. To avoid the algorithm becoming too specialized to one set of data, it’s trained in a way that takes into account what we know about the underlying structure of the problem. This new approach is shown to be better than other methods at finding the best solution when there are many variables. |
Keywords
» Artificial intelligence » Neural network » Objective function » Optimization » Overfitting » Representation learning » Semi supervised
