Summary of Monde: Mixture Of Near-data Experts For Large-scale Sparse Models, by Taehyun Kim et al.
MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Models
by Taehyun Kim, Kwanseok Choi, Youngmock Cho, Jaehoon Cho, Hyuk-Jae Lee, Jaewoong Sim
First submitted to arxiv on: 29 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Mixture of Near-Data Experts (MoNDE) solution efficiently enables Mixture-of-Experts large language models (LLM) inference by reducing the volume of parameter movement. MoNDE transfers only the “hot” experts to the GPU, computing the remaining “cold” experts inside host memory devices. This approach replaces massive expert parameter transfers with small activation transfers, resulting in significant speedups over existing frameworks for both encoder and decoder operations. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Mixture-of-Experts large language models need a lot of memory to work well. But when they don’t have enough space on the computer’s GPU, they need to move some information from slower storage devices to the faster GPU. This is slow and takes up a lot of energy. To solve this problem, researchers created Mixture of Near-Data Experts (MoNDE), which moves only the most important information to the GPU while keeping other parts on the slower device. By doing so, MoNDE makes it much faster and more efficient for language models to process information. |
Keywords
» Artificial intelligence » Decoder » Encoder » Inference » Mixture of experts
