Summary of Uniq: Offline Inverse Q-learning For Avoiding Undesirable Demonstrations, by Huy Hoang et al.
UNIQ: Offline Inverse Q-learning for Avoiding Undesirable Demonstrations
by Huy Hoang, Tien Mai, Pradeep Varakantham
First submitted to arxiv on: 10 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel approach to offline learning, focusing on avoiding undesirable demonstrations rather than imitating expert ones. By formulating the problem as maximizing a statistical distance between the learning policy and the undesirable policy, the authors create a new training objective that requires a novel algorithm. The proposed algorithm, UNIQ, is built upon the inverse Q-learning framework and frames the learning task as a cooperative process. The method leverages unlabeled data for practical training and outperforms state-of-the-art baselines on standard benchmark environments. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper helps us learn better without doing bad things. Normally, we try to copy good behavior, but what if we want to avoid bad actions? This is the problem the authors solve by creating a new way of learning that focuses on not doing unwanted things. They make a special algorithm called UNIQ that can use data we don’t need labels for and does better than others in tests. |
