Summary of Picle: Pseudo-annotations For In-context Learning in Low-resource Named Entity Detection, by Sepideh Mamooler et al.
PICLe: Pseudo-Annotations for In-Context Learning in Low-Resource Named Entity Detection
by Sepideh Mamooler, Syrielle Montariol, Alexander Mathis, Antoine Bosselut
First submitted to arxiv on: 16 Dec 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper explores In-Context Learning (ICL) for Large Language Models (LLMs), a method enabling models to perform tasks using few demonstrations. However, the effectiveness of ICL depends on the choice of demonstrations, and it remains unclear which attributes enable in-context generalization. The authors conduct a perturbation study on ICL for low-resource Named Entity Detection (NED) and surprisingly find that partially correct annotated entity mentions can be as effective for task transfer as fully correct demonstrations. Based on these findings, they propose Pseudo-annotated In-Context Learning (PICLe), a framework leveraging LLMs to annotate many demonstrations in a zero-shot first pass. PICLe then clusters these synthetic demonstrations, samples specific sets of in-context demonstrations from each cluster, and predicts entity mentions using each set independently. The authors evaluate PICLe on five biomedical NED datasets and show that it outperforms ICL in low-resource settings where limited gold examples can be used as in-context demonstrations. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper is about how to help Large Language Models learn new tasks quickly. Normally, these models need lots of training data to do well, but this method allows them to adapt to new tasks using just a few examples. The authors discovered that even if the examples aren’t perfect, they can still be very helpful for learning. They propose a new way to use these imperfect examples called Pseudo-annotated In-Context Learning (PICLe). This approach uses computer models to label many examples at once, and then it chooses the best ones to train on. The authors tested PICLe on five different datasets and found that it works better than previous methods in situations where there isn’t much training data. |
Keywords
» Artificial intelligence » Generalization » Zero shot
