Summary of Mgda: Model-based Goal Data Augmentation For Offline Goal-conditioned Weighted Supervised Learning, by Xing Lei et al.
MGDA: Model-based Goal Data Augmentation for Offline Goal-conditioned Weighted Supervised Learning
by Xing Lei, Xuetao Zhang, Donglin Wang
First submitted to arxiv on: 16 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Recently, Goal-Conditioned Weighted Supervised Learning (GCWSL) algorithms have revolutionized offline goal-conditioned reinforcement learning. GCWSL optimizes a lower bound of the goal-conditioned RL objective and has demonstrated impressive performance across various goal-reaching tasks. However, prior research identified a critical limitation: the lack of trajectory stitching capabilities. To address this, new goal data augmentation strategies were proposed to enhance these methods. Despite progress, existing techniques struggle to sample suitable augmented goals for GCWSL effectively. This paper proposes a Model-based Goal Data Augmentation (MGDA) approach that leverages a learned dynamics model to sample more suitable augmented goals. MGDA incorporates the local Lipschitz continuity assumption within the learned model to mitigate compounding errors. Empirical results show that MGDA significantly enhances GCWSL performance on state-based and vision-based maze datasets, surpassing previous goal data augmentation techniques. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine trying to get a robot to reach a specific goal without actually being there. That’s the challenge of offline reinforcement learning. Recently, new algorithms called GCWSL have shown promise in solving this problem. However, these algorithms have a limitation: they can’t stitch together different paths to reach the goal efficiently. To fix this, researchers proposed a way to generate more suitable goals for these algorithms. But this approach wasn’t perfect either. This paper introduces a new method that uses a learned model to generate better goals and improve stitching capabilities. The results are impressive – it works well on both virtual maze-like environments and real-world vision-based systems. |
Keywords
* Artificial intelligence * Data augmentation * Reinforcement learning * Supervised
