Summary of Knowledge Distillation From Language-oriented to Emergent Communication For Multi-agent Remote Control, by Yongjun Kim et al.
Knowledge Distillation from Language-Oriented to Emergent Communication for Multi-Agent Remote Control
by Yongjun Kim, Sejin Seo, Jihong Park, Mehdi Bennis, Seong-Lyun Kim, Junil Choi
First submitted to arxiv on: 23 Jan 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Information Theory (cs.IT); Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper compares two approaches for emergent communication in multi-agent deep reinforcement learning (MADRL) and language-oriented semantic communication (LSC) empowered by a pre-trained large language model (LLM). The first approach, emergent communication (EC), uses MADRL to learn from multimodal input data, while the second approach, LSC, relies on an LLM for inference. However, EC incurs high training costs and struggles with multimodal data, whereas LSC has high inference computing costs due to the large size of the LLM. To address these bottlenecks, the authors propose a novel framework called language-guided EC (LEC) that uses knowledge distillation (KD) to guide EC training using LSC. Simulations show that LEC achieves faster travel times and avoids areas with poor channel conditions, while also speeding up MADRL training convergence by up to 61.8% compared to EC. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper compares two ways of teaching computers to communicate. The first way uses a type of AI called deep reinforcement learning to learn from different types of data, like location and channel maps. The second way uses a pre-trained language model to help make decisions. However, the first way takes a long time to train and gets confused when dealing with multiple types of data, while the second way is slow because it needs to process so much information. To fix these problems, the authors created a new approach that combines the two methods. This approach helps computers learn faster and make better decisions. |
Keywords
* Artificial intelligence * Inference * Knowledge distillation * Language model * Large language model * Reinforcement learning
