Summary of Soft Prompt Tuning For Cross-lingual Transfer: When Less Is More, by Fred Philippy et al.
Soft Prompt Tuning for Cross-Lingual Transfer: When Less is More
by Fred Philippy, Siwen Guo, Shohreh Haddadan, Cedric Lothritz, Jacques Klein, Tegawendé F. Bissyandé
First submitted to arxiv on: 6 Feb 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Soft Prompt Tuning (SPT) method adapts pre-trained language models to specific tasks by inserting learnable embeddings at the input layer. This paper investigates SPT’s potential for cross-lingual transfer, focusing on keeping model parameters frozen while training only the soft prompt. The approach reduces computational costs and storage overhead, enhancing performance for linguistically distant languages. Factors like prompt length and reparameterization are explored to understand their impact on cross-lingual transfer. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary SPT is a way to make language models better at specific tasks without changing the model itself. This paper looks at how well SPT works when moving from one language to another that’s very different. They found that by only adjusting the “soft prompts” and leaving the rest of the model alone, they can get good results while using less computer power and storage. They also looked at what makes this process work better or worse. |
Keywords
» Artificial intelligence » Prompt
