Summary of Value-based Deep Multi-agent Reinforcement Learning with Dynamic Sparse Training, by Pihe Hu et al.
Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Training
by Pihe Hu, Shaolong Li, Zhuoran Li, Ling Pan, Longbo Huang
First submitted to arxiv on: 28 Sep 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach to accelerate training and compress models in Multi-Agent Reinforcement Learning (MARL) is proposed. MARL typically relies on neural networks with numerous parameters, leading to substantial computational overhead. To address this challenge, the authors introduce Dynamic Sparse Training (DST), which has shown promise in deep supervised learning tasks. However, a direct adoption of DST fails to yield satisfactory MARL agents, highlighting the need for innovative solutions. The proposed Multi-Agent Sparse Training (MAST) framework enhances value learning by improving reliability of learning targets and sample distribution. MAST incorporates the Soft Mellowmax Operator with a hybrid TD-() schema to establish dependable learning targets. It also employs a dual replay buffer mechanism to enhance training samples. Additionally, gradient-based topology evolution is used to exclusively train multiple MARL agents using sparse networks. Experimental results demonstrate significant reductions in redundancy of up to 20in Floating Point Operations (FLOPs) for both training and inference with less than 3% performance degradation. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary MARL helps machines learn from each other, but it uses lots of computer power. This paper suggests a way to make it faster and use less energy. Right now, MARL models are like super-complex computers that take forever to train. The authors want to fix this by using a technique called dynamic sparse training (DST). It works for simple tasks, but not for complex ones like MARL. So they came up with a new approach called MAST (Multi-Agent Sparse Training). MAST helps machines learn faster and more efficiently by making sure they get the right information. It also uses special tools to make the learning process smoother. The authors tested MAST on different MARL models and found that it can speed up training by 20 times while still keeping performance good. |
Keywords
» Artificial intelligence » Inference » Reinforcement learning » Supervised
