Summary of Off-dynamics Reinforcement Learning Via Domain Adaptation and Reward Augmented Imitation, by Yihong Guo et al.
Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitation
by Yihong Guo, Yixuan Wang, Yuanyuan Shi, Pan Xu, Anqi Liu
First submitted to arxiv on: 15 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Robotics (cs.RO)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed paper tackles the challenge of training policies in source domains for deployment in target domains under dynamics shifts. Previous work has attempted to address this issue by training on the source domain with modified rewards derived from matching distributions between the source and target optimal trajectories. However, these methods only ensure that the learned policy’s behavior resembles target optimal policies, but do not guarantee optimal performance when deployed to the target domain. The authors propose a new approach, Domain Adaptation and Reward Augmented Imitation Learning (DARAIL), which utilizes reward modification for domain adaptation and generative adversarial imitation learning from observation (GAIfO) with a reward-augmented estimator. Theoretically, an error bound is presented under mild assumptions regarding the dynamics shift. Empirically, DARAIL outperforms pure modified reward methods and other baselines in off-dynamics environments. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper tries to solve a tricky problem where you train a policy in one place but want it to work well in another place with different rules. Some people have tried using special rewards to make the training process better, but this doesn’t always work. The authors are proposing a new way to do this that combines two ideas: making sure the trained policy is good at imitating the best possible behavior in the target domain, and also making sure the policy is well-prepared for the new environment. |
Keywords
» Artificial intelligence » Domain adaptation
