Summary of Propd: Dynamic Token Tree Pruning and Generation For Llm Parallel Decoding, by Shuzhang Zhong et al.
ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding
by Shuzhang Zhong, Zebin Yang, Meng Li, Ruihao Gong, Runsheng Wang, Ru Huang
First submitted to arxiv on: 21 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The recent advancements in generative large language models (LLMs) have significantly boosted the performance in natural language processing tasks. However, their efficiency is hampered by the inherent limitations in autoregressive token generation. To address this issue, we propose ProPD, an efficient LLM parallel decoding framework based on dynamic token tree pruning and generation. This framework features an advanced early pruning mechanism to efficiently eliminate unpromising token sequences and introduces a dynamic token tree generation algorithm to balance computation and parallelism. We verify ProPD across various datasets, LLMs, and batch sizes, demonstrating consistent performance improvements of 1.1-3.2x compared to existing decoding algorithms. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary ProPD is a new way for computers to understand and generate human-like language. Right now, computers can only create text one word at a time, which takes a lot of time and energy. ProPD helps by letting computers work on many words at once, making it much faster and more efficient. This technology can be used in many areas like language translation, chatbots, and even self-driving cars. It’s an important step forward in helping computers communicate with us more naturally. |
Keywords
* Artificial intelligence * Autoregressive * Natural language processing * Pruning * Token * Translation
