Summary of Learning Goal-conditioned Policies From Sub-optimal Offline Data Via Metric Learning, by Alfredo Reichlin et al.
Learning Goal-Conditioned Policies from Sub-Optimal Offline Data via Metric Learning
by Alfredo Reichlin, Miguel Vasco, Hang Yin, Danica Kragic
First submitted to arxiv on: 16 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed MetricRL method tackles the challenge of learning optimal behavior from sub-optimal datasets for goal-conditioned offline reinforcement learning. By utilizing metric learning to approximate the optimal value function, the approach addresses sparse rewards, invertible actions, and deterministic transitions. The distance monotonicity property ensures that representations recover optimality, and an optimization objective leads to this property. The method guides policy learning in an actor-critic fashion. Experimentally, MetricRL outperforms state-of-the-art goal-conditioned RL methods by consistently estimating optimal behaviors from severely sub-optimal offline datasets without suffering from out-of-distribution estimation errors. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper proposes a new way to learn good decisions from imperfect data when we’re trying to get the best outcome. They use something called “metric learning” to make an estimate of what the best decision would be, even if our current data is not very good. This helps avoid mistakes and gets better results. The approach is tested and shown to work well in different situations, outperforming previous methods. |
Keywords
* Artificial intelligence * Optimization * Reinforcement learning
