Summary of A Tensor Network Implementation Of Multi Agent Reinforcement Learning, by Sunny Howard
A Tensor Network Implementation of Multi Agent Reinforcement Learning
by Sunny Howard
First submitted to arxiv on: 8 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Multiagent Systems (cs.MA)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper demonstrates how tensor networks (TNs) can be used to represent the expected return of a multi-agent reinforcement learning (MARL) task. Building on previous work that showed TNs can model single-agent finite Markov decision processes, this study extends the approach to multiple agents. The key challenge is addressing the curse of dimensionality, where the number of possible trajectories grows exponentially with the number of agents. To tackle this issue, established optimization and decomposition techniques specific to TNs are applied to find an efficient representation. The method is tested on a 2-agent random walker example, where it successfully optimizes the policy using a density matrix renormalization group (DMRG) technique. Additionally, an exact decomposition technique is demonstrated, reducing the number of tensor elements by 97.5% without sacrificing information. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper shows how to use special networks called tensor networks to help computers learn from multiple agents working together. This is important because when there are many agents, it can get very complicated and slow down the learning process. The researchers used techniques that are already known for working with these types of networks to make sure they got an efficient answer. They tested their method on a simple example where two agents were moving around randomly, and it worked well. This could be useful in many areas, such as games or self-driving cars. |
Keywords
* Artificial intelligence * Optimization * Reinforcement learning
