Summary of Adversarially Robust Decision Transformer, by Xiaohang Tang et al.
Adversarially Robust Decision Transformer
by Xiaohang Tang, Afonso Marques, Parameswaran Kamalaruban, Ilija Bogunovic
First submitted to arxiv on: 25 Jul 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The Decision Transformer (DT) has achieved strong performance in offline learning tasks by leveraging the Transformer architecture for sequential decision-making. However, it can be non-robust in adversarial environments as its return is dependent on the strategies of both the decision-maker and adversary. To address this, the authors propose a worst-case-aware RvS algorithm called Adversarially Robust Decision Transformer (ARDT), which learns and conditions the policy on in-sample minimax returns-to-go. ARDT aligns the target return with the worst-case return learned through minimax expectile regression, enhancing robustness against powerful test-time adversaries. In experiments conducted on sequential games and continuous adversarial RL environments, ARDT demonstrates superior robustness to powerful test-time adversaries and attains higher worst-case returns compared to contemporary DT methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about a new way to make decision-making machines more reliable in tricky situations. Right now, some of these machines can be tricked into making bad choices by an opponent trying to manipulate them. To fix this, the researchers created a new algorithm that helps the machine learn from good and bad examples to make better decisions even when things get tough. |
Keywords
* Artificial intelligence * Regression * Transformer
