Summary of M-hof-opt: Multi-objective Hierarchical Output Feedback Optimization Via Multiplier Induced Loss Landscape Scheduling, by Xudong Sun et al.
M-HOF-Opt: Multi-Objective Hierarchical Output Feedback Optimization via Multiplier Induced Loss Landscape Scheduling
by Xudong Sun, Nutan Chen, Alexej Gossmann, Matteo Wohlrapp, Yu Xing, Carla Feistner, Emilio Dorigatt, Felix Drost, Daniele Scarcella, Lisa Beer, Carsten Marr
First submitted to arxiv on: 20 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed probabilistic graphical model models the joint evolution of model parameters and multipliers, using a hypervolume-based likelihood to promote multi-objective descent in structural risk minimization. The model optimizes multiple objectives via a surrogate single-objective penalty loss with time-varying multipliers, equivalent to online scheduling of loss landscapes. The method hierarchically dispatches the multi-objective descent goal into constraint optimization sub-problems with shrinking bounds based on Pareto dominance. This approach forms a closed-loop system that avoids excessive memory requirements and computational burden, while being robust against controller hyperparameter variation. It is demonstrated on domain generalization tasks with multi-dimensional regularization losses. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A new way to optimize machine learning models is proposed. Instead of trying to find the best model parameters, this method focuses on finding a good balance between different goals. This is done by using a special type of mathematical framework called a probabilistic graphical model. The model uses a clever trick to reduce the complexity of the optimization problem, making it more efficient and robust. This approach has been tested on challenging tasks that require learning from data in multiple domains, and it shows promising results. |
Keywords
* Artificial intelligence * Domain generalization * Hyperparameter * Likelihood * Machine learning * Optimization * Regularization
