Summary of Extrinsicaly Rewarded Soft Q Imitation Learning with Discriminator, by Ryoma Furuyama et al.
Extrinsicaly Rewarded Soft Q Imitation Learning with Discriminator
by Ryoma Furuyama, Daiki Kuyoshi, Satoshi Yamane
First submitted to arxiv on: 30 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A machine learning algorithm that combines Behavioral Cloning and soft Q-learning with constant rewards, called Soft Q imitation learning (SQIL), has been shown to learn efficiently. However, this method can be prone to distribution shift. To address this issue, a new algorithm, Discriminator Soft Q Imitation Learning (DSQIL), is proposed by adding a reward function based on adversarial inverse reinforcement learning that rewards the agent for performing actions in states similar to the demo. The goal of DSQIL is to learn from only a few expert data and make the imitation learning process more robust. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A new way to teach machines to imitate human behavior has been developed. This method, called Soft Q imitation learning (SQIL), works well when there’s not much data available. However, it can be tricky to use in situations where the environment is changing or where the reward system is complex. To make this algorithm better, a new version called Discriminator Soft Q Imitation Learning (DSQIL) has been created. This updated method adds a special way of rewarding the machine for doing actions that are similar to what a human would do. The goal is to make it easier to teach machines how to imitate humans. |
Keywords
* Artificial intelligence * Machine learning * Reinforcement learning
