Summary of Pocketllm: Enabling On-device Fine-tuning For Personalized Llms, by Dan Peng et al.
PocketLLM: Enabling On-Device Fine-Tuning for Personalized LLMs
by Dan Peng, Zhihui Fu, Jun Wang
First submitted to arxiv on: 1 Jul 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed approach enables the fine-tuning of large language models (LLMs) on mobile devices without sacrificing data privacy. The key innovation is the use of derivative-free optimization techniques to optimize model parameters, which overcomes the memory constraints imposed by traditional derivative-based methods. This is achieved by leveraging the RoBERTa-large and OPT-1.3B models, which can be fine-tuned locally on resource-constrained devices like the OPPO Reno 6 smartphone with as little as 4GB or 6.5GB of memory, respectively. The feasibility of this approach paves the way for personalized LLMs on mobile devices while ensuring data privacy. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary On-device language models can be fine-tuned without sharing sensitive data. This is important because many people use their phones to chat with friends and family, but they might not want to share all their conversations. To make this work, the researchers developed a new way of updating the model’s parameters that doesn’t require as much memory. They tested it on two big language models and found that it works well even on smaller devices like smartphones. |
Keywords
* Artificial intelligence * Fine tuning * Optimization
