Summary of Transition Constrained Bayesian Optimization Via Markov Decision Processes, by Jose Pablo Folch et al.
Transition Constrained Bayesian Optimization via Markov Decision Processes
by Jose Pablo Folch, Calvin Tsay, Robert M Lee, Behrang Shafei, Weronika Ormaniec, Andreas Krause, Mark van der Wilk, Ruth Misener, Mojmír Mutný
First submitted to arxiv on: 13 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Bayesian optimization for black-box functions faces limitations when query constraints are present, such as local movement restrictions or transitions influencing measurement accuracy. This work extends classical Bayesian optimization by incorporating Markov Decision Processes (MDPs), leveraging reinforcement learning to iteratively solve a linearized utility function and obtain a policy that plans ahead. The resulting policy is history-dependent and non-Markovian, showcasing applications in chemical reactor optimization, informative path planning, machine calibration, and synthetic examples. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper takes Bayesian optimization to the next level by considering real-life constraints that weren’t accounted for before. Instead of being able to query any part of a search space, the algorithm now needs to plan ahead based on previous queries. This is especially important in fields like physics, where measurements can be influenced by what happened earlier. The new approach uses reinforcement learning and Markov Decision Processes to find an optimal policy that takes these constraints into account. |
Keywords
* Artificial intelligence * Optimization * Reinforcement learning
