Summary of Dynamic Anisotropic Smoothing For Noisy Derivative-free Optimization, by Sam Reifenstein et al.
Dynamic Anisotropic Smoothing for Noisy Derivative-Free Optimization
by Sam Reifenstein, Timothee Leleu, Yoshihisa Yamamoto
First submitted to arxiv on: 2 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Optimization and Control (math.OC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel algorithm is proposed that combines ball smoothing and Gaussian smoothing for noisy derivative-free optimization by accounting for the heterogeneous curvature of the objective function. The algorithm dynamically adapts the shape of the smoothing kernel to approximate the Hessian of the objective function around a local optimum, reducing error in estimating the gradient from noisy evaluations through sampling. This approach demonstrates improved performance compared to existing state-of-the-art methods in tuning NP-hard combinatorial optimization solvers. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper proposes an algorithm that helps optimize functions by smoothing out errors and adapting to the shape of the function. It does this by adjusting a special kind of averaging process, called a “smoothing kernel”, to match the shape of the function around its best points. This makes it better at finding the right answer. The researchers tested their method on some simple problems and found that it worked really well. |
Keywords
» Artificial intelligence » Objective function » Optimization
