Summary of Language Model Cascades: Token-level Uncertainty and Beyond, by Neha Gupta et al.
Language Model Cascades: Token-level uncertainty and beyond
by Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar
First submitted to arxiv on: 15 Apr 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates cascading strategies for generative language models, aiming to improve the cost-quality tradeoffs by deferring challenging instances to a large model while invoking a small model for easier ones. The authors focus on predict sequence uncertainty, but find that it suffers from length bias and propose incorporating token-level uncertainty through learned post-hoc deferral rules. Experimental results on natural language benchmarks with FLAN-T5 models show that the proposed approach outperforms simple aggregation strategies, achieving improved cost-quality tradeoffs. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper looks at how to make language models more efficient by letting a smaller model handle easy tasks and leaving harder ones for a larger model. The authors test different ways of predicting which instances are hard or easy and find that the current method has some problems. They propose a new approach that uses information from both the small and large models, which helps improve performance on natural language tasks. |
Keywords
» Artificial intelligence » T5 » Token
