Summary of How to Leverage Diverse Demonstrations in Offline Imitation Learning, by Sheng Yue et al.
How to Leverage Diverse Demonstrations in Offline Imitation Learning
by Sheng Yue, Jiani Liu, Xingyuan Hua, Ju Ren, Sen Lin, Junshan Zhang, Yaoxue Zhang
First submitted to arxiv on: 24 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Medium Difficulty summary: Offline Imitation Learning (IL) with imperfect demonstrations has gained attention due to the scarcity of expert data in many domains. A key challenge is extracting positive behaviors from noisy data. Current approaches select data based on state-action similarity, overlooking information in diverse state-actions that deviate from experts. This paper introduces a simple yet effective data selection method identifying positive behaviors by their resultant states, leveraging dynamics information and extracting both expert and beneficial diverse behaviors. Additionally, it proposes a lightweight behavior cloning algorithm capable of utilizing the selected data correctly. The experiments evaluate the method on complex offline IL benchmarks, including continuous-control and vision-based tasks. The results show that our method achieves state-of-the-art performance, outperforming existing methods on 20/21 benchmarks, typically by 2-5x, while maintaining a comparable runtime to Behavior Cloning (BC). |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Low Difficulty summary: This research helps robots learn from imperfect examples, which is important because we don’t always have perfect data. The main problem is that current methods only look at what the expert does and not what happens as a result. This new method looks at both what the expert does and what happens afterwards to find good actions. It also has an algorithm that can use this information correctly. The results show that this method performs better than others on 20 out of 21 tasks, while taking about the same amount of time. |
Keywords
* Artificial intelligence * Attention
