Summary of Detail: Task Demonstration Attribution For Interpretable In-context Learning, by Zijian Zhou et al.
DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning
by Zijian Zhou, Xiaoqiang Lin, Xinyi Xu, Alok Prakash, Daniela Rus, Bryan Kian Hsiang Low
First submitted to arxiv on: 22 May 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores In-context Learning (ICL), a technique that enables pre-trained transformer-based language models to quickly learn specific tasks with few “task demonstrations” without updating their parameters. ICL is distinct from conventional machine learning, requiring new approaches for interpretation. The authors propose DETAIL, an influence function-based attribution technique, to address the characteristics of ICL. They empirically verify DETAIL’s effectiveness and demonstrate its applicability in real-world scenarios through demonstration reordering and curation. Additionally, they show that DETAIL’s attribution scores obtained from white-box models are transferable to black-box models, improving model performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about a new way for computers to learn new tasks quickly without needing lots of data or training time. It uses special language models called transformers and shows how they can learn by just seeing a few examples of what the task looks like. The authors also develop a tool to help us understand why these models make certain decisions, which is important for making sure they don’t get biased or unfair. They test this tool on some real-world problems and show that it works well. |
Keywords
» Artificial intelligence » Machine learning » Transformer
