Summary of Melting Point: Mobile Evaluation Of Language Transformers, by Stefanos Laskaridis et al.
MELTing point: Mobile Evaluation of Language Transformers
by Stefanos Laskaridis, Kleomenis Katevas, Lorenzo Minto, Hamed Haddadi
First submitted to arxiv on: 19 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Transformers have transformed the machine learning landscape, but their runtime requirements have hindered their widespread deployment on mobile devices. The paper explores the current state of executing Large Language Models (LLMs) on device, creating an automation infrastructure called MELT to support headless execution and benchmarking on various models, devices, and frameworks. The authors evaluate popular instruction-fine-tuned LLMs and analyze their performance, energy efficiency, and accuracy across different state-of-the-art models. Results highlight the performance heterogeneity across targets and show that LLM inference is memory-bound. Quantization reduces memory requirements but affects accuracy. The study demonstrates the challenges in executing LLMs continuously on device due to energy footprint and thermal behavior. The authors conclude that NPU acceleration, framework-hardware co-design, and offloading might be key to efficient standalone execution. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine a future where computers can think and learn like humans. Transformers are a type of artificial intelligence that has made great progress in recent years. However, they require a lot of power to run, which makes them difficult to use on mobile devices like smartphones. This paper explores ways to make these powerful AI models work better on smaller devices. The authors created a special tool called MELT to test different AI models and see how well they perform on various devices. They found that some AI models are much faster and more efficient than others, but there are still challenges in making them run continuously without overheating or using too much power. Overall, this research helps us understand the potential of these powerful AI models and what it takes to make them work effectively on small devices. |
Keywords
* Artificial intelligence * Inference * Machine learning * Quantization
